Use of open-text responses to recode categorical survey data on postpartum contraception use among women in the United States: A mixed-methods inquiry of Pregnancy Risk Assessment Monitoring System data

Nicole K Richards; Christopher P Morley; Martha A Wojtowycz; Erin Bevec; Brooke A Levandowski

doi:10.1371/journal.pmed.1003878

. 2022 Jan 5;19(1):e1003878. doi: 10.1371/journal.pmed.1003878

Use of open-text responses to recode categorical survey data on postpartum contraception use among women in the United States: A mixed-methods inquiry of Pregnancy Risk Assessment Monitoring System data

Nicole K Richards ^1,^2,^3,^*, Christopher P Morley ^2,³, Martha A Wojtowycz ^3,⁴, Erin Bevec ^5,⁶, Brooke A Levandowski ^3,⁵

Editor: Sarah J Stock⁷

PMCID: PMC8775313 PMID: 34986158

Abstract

Background

Postpartum contraception prevents unintended pregnancies and short interpregnancy intervals. The Pregnancy Risk Assessment Monitoring System (PRAMS) collects population-based data on postpartum contraception nonuse and reasons for not using postpartum contraception. In addition to quantitative questions, PRAMS collects open-text responses that are typically left unused by secondary quantitative analyses. However, abundant preexisting open-text data can serve as a resource for improving quantitative measurement accuracy and qualitatively uncovering unexpected responses. We used PRAMS survey questions to explore unprompted reasons for not using postpartum contraception and offer insight into the validity of categorical responses.

Methods and findings

We used 31,208 categorical 2012 PRAMS survey responses from postpartum women in the US to calculate original prevalences of postpartum contraception use and nonuse and reasons for contraception nonuse. A content analysis of open-text responses systematically recoded data to mitigate survey bias and ensure consistency, resulting in adjusted prevalence calculations and identification of other nonuse themes. Recoded contraception nonuse slightly differed from original reports (21.5% versus 19.4%). Both calculations showed that many respondents reporting nonuse may be at a low risk for pregnancy due to factors like tubal ligation or abstinence. Most frequent nonuse reasons were not wanting to use birth control (27.1%) and side effect concerns (25.0%). Other open-text responses showed common themes of infertility, and breastfeeding as contraception. Comparing quantitative and qualitative responses revealed contradicting information, suggesting respondent misinterpretation and confusion surrounding the term “pregnancy prevention.” Though this analysis may be limited by manual coding error and researcher biases, we avoided coding exhaustion via 1-hour coding periods and validated reliability through intercoder kappa scores.

Conclusions

In this study, we observed that respondents reporting contraception nonuse often described other methods of pregnancy prevention and contraception barriers that were not included in categorical response options. Open-text responses shed light on a more comprehensive list of pregnancy prevention methods and nonuse options. Our findings contribute to survey questions that can lead to more accurate depiction of postpartum contraceptive behavior. Additionally, future use of these qualitative methods may be used to improve other health behavior survey development and resulting data.

In a mixed-methods inquiry, Nicole Richards and colleagues investigate how open-text responses can provide insight into categorical reasons for postpartum contraception use and non-use among women in the US.

Author summary

Why was this study done?

Survey respondents can occasionally provide open-ended answers that may be overlooked during quantitative data analysis.
This study was conducted to study how qualitative inquiry of open-text responses to a survey question related to postpartum contraception use can be used to provide insight into data accuracy within quantitative surveys.

What did the researchers do and find?

Using a mixed-method approach, we calculated descriptive statistics from Pregnancy Risk Assessment Monitoring System (PRAMS) survey responses from 31,208 postpartum women in the US who provided categorized reasons for not using contraception and qualitatively analyzed respondents’ adjacent open-ended written answers to uncover more detailed information about respondents’ postpartum contraception behavior and interpretations of the survey question.
Through systematic recategorization, we found that qualitative responses added to and altered quantitative contraception behavior reports and that reasons for contraception nonuse are incompletely captured by existing response categories.

What do these findings mean?

These findings suggest that researchers may be able to gain a better understanding of postpartum pregnancy prevention by using mixed-method analyses on categorical and open-ended responses.
While qualitative inquiry may yield subjective interpretation, rigorous pragmatic data management can strengthen survey questions through open-text responses not captured in existing quantitative data.
Family planning professionals may be able to obtain significant and detailed information about contraceptive behavior from open-ended responses, and this information may be useful to adjust clinical approaches to postpartum contraception conversations.

Introduction

Postpartum contraception uptake mitigates unintended pregnancies and inadequate birth spacing, playing a large role in preventing poor birth outcomes and promoting healthy mothers, infants, and families [1,2]. Many factors influence contraception decisions and access, and, therefore, not all people that want to prevent pregnancy use a contraception method [3]. Some primary care clinicians report assumptions that the patient will initiate contraception conversations, and both patients and providers report struggling with contraception counseling due to lack of contraception knowledge and comfort [4,5]. Further, health literacy and numeracy can impact patients’ reproductive health knowledge, behaviors, and outcomes [6]. Specific to this study, low health literacy has been linked to lack of contraception and fertility knowledge, including mechanisms of contraception methods [7] and contraception adherence [8].

The nature of self-reported datasets and secondary analysis leaves much room for information bias due to social desirability and health literacy levels. The potential for misunderstanding questions and response options impacts how respondents understand and reply to questions, which, in turn, may influence analyses that only utilize quantitative data and do not utilize additional qualitative data collected. The Pregnancy Risk Assessment Monitoring System (PRAMS) is a Centers for Disease Control and Prevention (CDC)-facilitated population-based surveillance system, which supports maternal and child health research [9]. An example of information bias is found in one PRAMS analysis, which found that among women reporting a mistimed or unwanted pregnancy as a result of contraception nonuse, 30% reported that they did not use a method because they did not mind having a pregnancy [10]. Without deeper investigation, it is not intuitive that someone would report both an unwanted pregnancy and not minding a pregnancy at the same time. Similarly, our original study sought to explore respondents’ reported postpartum contraception behavior among those reporting an unintended pregnancy, including reasons why women do not actively choose subsequent pregnancy prevention. The objective of these analyses was to identify how responses to PRAMS survey questions match respondents’ consistent and accurate (reliability and validity) interpretations through the additional analysis of qualitative observations [11,12].

Methods

PRAMS, a surveillance project run by the CDC and state and local health departments, covers approximately 81% of US births by collecting state birth certificate data and including both universal core questions and site-specific questions about maternal attitudes and behaviors related to pregnancy [9]. Postpartum women are identified through birth certificate data and mailed questionnaires 2 to 4 months after delivery. Potential participants are followed up via mail and phone to encourage higher response rates. PRAMS asks quantitative categorical questions, some of which include a response choice of “other,” which is followed by an open text box to fill in another response. Thus, these open-text responses were used to qualitatively evaluate quantitative survey questions. For these analyses, the core questionnaire of interest asks participants about their reasons for not preventing a subsequent pregnancy through categorical and open-ended options [13]. We chose a subset of Phase VII of PRAMS (2012 to 2015) because it includes the most comprehensive list of categorical choices (n = 9) compared to Phases V, VI, and VIII. PRAMS Phase VII respondents (n = 65,407) were chosen via recent birth certificate data; those with high-risk characteristics were selected at a higher rate to collect adequate amount of data on smaller populations. Phase VII’s response rate threshold was ≥60%. This study analyzes only the 2012 cycle as a standard sample that could both satisfy quantitative value and reach qualitative saturation.

We integrated qualitative and quantitative epistemologies to discover a pragmatic middle-ground reality rather than objective or subjective accounts (for example, fixed qualitative coding structures and interrater reliability) [14]. Balancing quantitative and qualitative rigor standards of sample size and theoretical saturation, we performed a secondary analysis of Phase VII PRAMS 2012 cycle (n = 31,208) to investigate contraception use/nonuse and reasons for nonuse. We first took a basic quantitative inventory of original response data percentages. We then used a qualitative thematic approach to develop our recategorization protocol. This protocol was used to standardize a subjective qualitative approach, creating a prospective plan to better fit the objective quantitative intentions of the PRAMS survey (Fig 1). Analysts documented the qualitative process with observational memos, including thoughts about sociolinguistics and phenomena occurring. Lastly, we used kappa scores, counts, and percentages to tie in qualitative epistemological findings with quantitative measures. This mixed-method lens enabled a richer understanding of the data while managing researcher biases. This research was exempt from review by Upstate Medical University’s Institutional Review Board (#1165109).

Fig 1 — Researchers used this tool to ensure consistent coding and recategorization of contraception behavior responses. NFP, natural family planning; PRAMS, Pregnancy Risk Assessment Monitoring System.

Original data analysis

To identify those that were and were not using contraception, we analyzed data from PRAMS core question 50: “Are you or your husband or partner doing anything now to keep from getting pregnant? Some things people do to keep from getting pregnant include using birth control pills, condoms, withdrawal, or natural family planning” (n = 31,208). We then explored possible postpartum contraception uptake barriers through core question 51: “What are your reasons or your partner’s reasons for not doing anything to keep from getting pregnant postpartum? Check all that apply.” In addition to structural and financial reasons for contraception nonuse, the term “barriers” includes personal constraints such as cultural and perception influence [15]. We calculated original postpartum contraception use prevalence (n = 30,637) and proportions of each categorical nonuse reason including abstinence, wants pregnancy, does not want to use, side effects, partner does not want to use, access problems, tubal ligation, vasectomy, currently pregnant, and other reason (n = 6,044) using SPSS software version 26 (IBM). Lastly, we reviewed case-by-case consistency between quantitative data and corresponding qualitative reports in Excel 2016 to improve categorization [16].

Recoding variables

We followed traditional qualitative underpinnings of emerging and iterative data analysis to develop steps of a systematic approach to data recategorization [12]. Researchers read open-text responses for data familiarization, open-coded for preliminary trends, deliberated on a uniform coding protocol, and finalized the systematic recoding process (Fig 1). Data were manually recoded following predetermined steps. First, we reviewed the postpartum contraception use (y/n) variable according to the question framework, which did not categorize abstinence, vasectomy, and tubal ligation as contraception methods. We recategorized respondents as using contraception if the open-text response indicated pregnancy prevention through examples given in PRAMS core question 50 (birth control, condoms, withdrawal, or natural family planning). Next, we deleted categorical nonuse reasons if they were contradicted by open-text responses and input categorical reasons if open-text responses aligned with predetermined categories. For example, if abstinence category was not chosen but a respondent wrote “not having sex,” we checked the abstinence category. In addition, specific open-text responses that fit within larger categorical themes were appropriately coded in their corresponding category. For example, we categorized financial barriers or waiting for a contraception method as “having problems getting birth control,” religious or cultural objections to birth control as “not wanting to use a method,” and currently okay with becoming pregnant again as “I want to become pregnant.” Using the agreed-upon codebook, original coding was conducted by one female author (NKR), creating a new variable for each code (Affordability; Beliefs; Breastfeeding: LAM*/No ovulation, Breastmilk Impact; Fertility Issues; Hysterectomy; Office Barriers; Same Sex Relationship; Situational Barriers; Medical Condition; Will Get a Method). We calculated number of respondents in each new category via SPSS descriptive statistics function. A separate female author (EB) recoded a random sample of 10% of all responses. A kappa score of agreement for interrater reliability was calculated for each variable. We calculated new unweighted and weighted prevalences to show a more accurate picture of postpartum contraception use and reasons for nonuse, using chi-squared tests to determine differences in variable distribution. Weighted percentages were calculated in Stata version 16.1 using survey weights and svy commands.

Qualitative analysis

We explored open-text responses in NVivo version 12 to identify common and specific nonuse explanations not captured by categorical options. We reached theme development through content analysis, allowing us to become familiar with the data, identify initial codes, and subsume codes into categories [17]. We then quantified each theme to illustrate frequencies and exemplified these phenomena through excerpts. This study is reported as per the Standards for Reporting Qualitative Research (SRQR) guideline (S1 Checklist) [18].

Results

Original response data

Raw reports of postpartum contraception use showed that 78.8% (n = 24,593/31,208) of postpartum women were using some form of postpartum contraception (Table 1). Further, many of those that reported no pregnancy prevention also reported controlling their fertility through abstinence (36.7%) or tubal ligation (13.2%) in the follow-up question. The most prominent reasons for not preventing pregnancy were personal preference to not use birth control (29.8%) and concerns with birth control side effects (27.9%). Respondents commonly chose more than one reason they were not using a pregnancy prevention method. Despite this question’s 9 categorical choices, 30% reported additional reasons for not preventing a subsequent pregnancy via open-text responses.

Table 1. Original and recoded postpartum contraception behavior (n = 31,208) and reasons for nonuse among PRAMS Phase VII (2012) respondents.

	Original Response Data			Recoded Sample
	n	%	Weighted %	n	%	Weighted %
Using Contraception Method
Yes	24,593	78.8	79.0	24,020	77.0	77.2^**
No	6,044	19.4	19.3	6,719	21.5	21.4^**
Missing	571	1.8	1.8	469	1.5	1.4
Nonuse Reasons^*
Abstinence	2,222	36.7	6.7	2,183	32.5	6.6^**
Want Pregnancy	990	16.4	3.2	1,029	15.3	3.4^**
Do not Want To Use Birth Control	1,801	29.8	6.0	1,818	27.1	6.1^**
Birth Control Side Effects	1,688	27.9	5.2	1,680	25.0	5.2^**
Partner’s Decision	733	12.1	2.4	721	10.7	2.4^**
Problems Getting Birth Control	271	4.5	0.8	370	5.5	1.1^**
Tubal Ligation	797	13.2	2.4	794	11.8	2.4^**
Vasectomy	104	1.7	3.1	101	1.5	3.1^**
Currently Pregnant	187	3.1	4.6	186	2.8	4.6^**
Other Barriers	1,569	30.0	5.2	1,356	20.2	4.4^**

Open in a new tab

*Percent denominators are those not using contraception (original response data: 6,044; recoded sample: 6,719).

**p-Value < 0.001.

PRAMS, Pregnancy Risk Assessment Monitoring System.

Instead of choosing a categorical reason for not preventing a postpartum pregnancy, respondents sometimes alternatively reported an open-text response that fit within the existing response categories. Similarly, other open-text responses included explanations of categories already chosen (for example, choosing abstinence and elaborating on why they were abstinent).

Though the “reasons for contraception nonuse” question asked respondents that were using contraception to skip this section, respondents using a method sometimes still answered this question. Some reported contraception use but provided reasons why they were not preventing pregnancy. In other cases, open-text responses described their current method (for example, IUD, implant, and birth control shot). Further, both the contraception user and nonuser groups reported practicing abstinence or having a tubal ligation procedure, suggesting a nonuniform understanding of pregnancy prevention methods.

Recoded data

We found high interrater reliability between coders (kappa range 0.82 to 1.0) and slight changes in the prevalence of postpartum contraception use and nonuse reasons after our manual recoding (Table 1). Overall, 675 more women were not using a PRAMS-defined contraception method than the original data showed (21.5% versus 19.4%). Moreover, a 9.8% decrease (n = 213) in the “other reason for nonuse” category illustrates a sizable group that should have skipped this question (that is, those that used contraception or did not indicate contraception status). Likewise, most recoded categorical prevalences decreased from original data reports. All recoded variables were statistically significantly different when weighted percentages were compared to original percentages.

Open-text qualitative themes

Open-text responses identified other nonuse themes in addition to prompted reasons for contraception nonuse (Table 2). We found that some women were preventing pregnancy through other contraceptive methods; 40 women did not require contraception because they were in a same-sex relationship, 32 had a hysterectomy after their last birth, and 197 had difficulty getting pregnant for several other reasons. Those with fertility problems often mentioned how long it took to conceive, conditions that impacted fertility, and fertility treatment methods. Others relied on not having a postpartum menstrual cycle (n = 100), with most of these women using lactation amenorrhea to prevent ovulation. Open-text responses showed frustration that these reasons were not listed as categorical options (for example, capitalizing words, using exclamation points, and suggesting survey question changes).

Table 2. Qualitative themes of postpartum contraception nonuse reasons among PRAMS Phase VII (2012) respondents.

Theme	Examples	n
Affordability	“No insurance,” “My insurance doesn’t believe in birth control!,” “Too expensive”	67
Beliefs	“Religious reasons,” “I don’t believe in birth control,” “All God’s plan”	90
Breastfeeding		164
LAM^*/No ovulation	“Breastfeeding exclusively,” “No menstruation due to breastfeeding,” “Nursing is a form of birth control”	100
Breastmilk Impact	“Birth control almost stopped my supply of Breastmilk,” “Chemicals/hormones breastfeeding”	31
Fertility Issues	“Took a really long time to get pregnant” “We used IVF”	197
Hysterectomy	“Womb removed,” “Hysterectomy after birth,” “Doctor took my uterus”	32
Office Barriers	“we were not informed,” “my doctor doesn’t prescribe birth control,” “signed up for it but still waiting”	56
Same-Sex Relationship	“female partner!,” “SAME-SEX PARTNER ^*maybe you should change your survey,” “I’m gay”	40
Situational Barriers	“I can’t find the right birth control,” “Haven’t had time,” “Stress,” “Transportation issues”	76
Medical Condition	“Blood pressure too high,” “Blood clot disorder,” “Can’t take hormonal due to certain meds”	36
Will Get a Method	“I am getting tubes tied,” “Waiting to get IUD put in,” “husband is scheduling a vasectomy”	115

Open in a new tab

*Lactation amenorrhea method.

IVF, in vitro fertilization; IUD, intrauterine device; PRAMS, Pregnancy Risk Assessment Monitoring System.

Women reported that they could not use a contraception method for a few common reasons. Approximately 36 women reported having medical conditions that limited their preferred birth control options (for example, heart and blood conditions). Similarly, breastfeeding mothers were concerned about birth control’s breastmilk impact (n = 31), such as hormones and decreased milk supply. Others chose not to use a method due to negative contraception beliefs (n = 90), which often stemmed from cultural and religious values.

Many participants intended to use a contraception method in the near future but were not currently preventing pregnancy for a number of reasons (n = 115). Affordability was a concern for 67 women, with several reporting no contraception coverage because they lost their insurance during the postpartum period or their insurance did not cover contraception. Further, 56 women reported office barriers, such as long waits for contraception counseling and procedure appointments, and clinician reluctance to provide a method. Further, 76 women faced situational barriers to contraception, including lack of transportation or time to attend an appointment.

Discussion

Our study shows that qualitative analyses of open-text survey responses can contribute to accurate data categorization and resulting analyses. Indeed, while several overall weighted percentages in each category were the same or only different by tenths of a percent, chi-squared tests indicate that the recoded categories were all statistically significantly different after this recoding exercise. The statistically significant differences suggest the importance of reexamining survey responses to improve data quality; however, it should be noted that the original and recoded findings yield similar measures of contraception use. Overall, these PRAMS questions illuminated an unclear understanding of the definition of pregnancy prevention in some instances, and this may have contributed to inconsistent answers based on respondent interpretation. Additionally, the nature of self-administered mailed surveys may have contributed to respondents missing skip patterns (that is, some that were preventing pregnancy mistakenly reported reasons why they were not preventing pregnancy). With many respondents reporting similar noncategorical reasons for contraception nonuse (for example, breastfeeding and fertility issues), additional categorical options could be useful to capture a more comprehensive picture of contraception behavior.

Using PRAMS categorical and open-text responses to identify contraception behaviors and barriers enabled the study team to collate population-based phenomena occurring among postpartum women. This study is unique in its ability to contribute qualitative information about a large sample of postpartum women’s contraception behavior. Further, the methods used establish qualitative data’s usefulness in large datasets. Qualitative thematic text analyses are already used to explore new ideas within open-ended survey questions [19]. That is, researchers may conduct qualitative methods on either (1) a stand-alone open-ended survey question or (2) a categorical survey question that is followed by a prompt to “explain” the categorical choice. However, this paper offers a detailed process of how qualitative and quantitative methods can be used in combination as a tool to strengthen already existing survey questions.

Our unique qualitative approach to this dataset identified misclassification bias, including miscategorized categorical responses and underlying contraception barriers not captured by original data analysis. We mitigated manual coding error and researcher biases by limiting coding to 1-hour periods and calculating kappa scores for codebook validation. The qualitative portion and systematic review of original data uncovered detailed information about respondent values and behavior rather than relying solely on self-reported quantitative data. Though we mitigated biases when possible, qualitative methods innately introduce researcher bias. Further, the use of these methods for survey improvement were only performed on one survey question within one national dataset, leaving a need for replication in other survey open-text datasets to better validate as a transferrable approach.

Based on our PRAMS analyses, we suggest collecting cleaner data, ensuring that questions like these are asked in a way that systematically categorizes contraception and uses simple language comprehendible to those with limited health literacy or language barriers. For example, the National Survey of Family Growth collects information about using contraception methods by asking yes–no questions about every method [20]. While we understand each survey’s scope and data collection method can contribute to the comprehensiveness of contraception use questions, we recommend open-text responses like this PRAMS question are thoroughly examined and cleaned prior to secondary analyses.

Specific to PRAMS question 50 and 51, we recommend asking all respondents “Are you doing any of the following to prevent pregnancy?”, providing a comprehensive list of all contraception methods with yes/no responses, and including a “none of the above” option. If a respondent chose none of the above, they would then answer the second question “What are your reasons for not preventing pregnancy?” Fig 2 displays an example of a comprehensive survey design approach to eliciting more accurate contraception behavior approaches.

Fig 2 — Revising current contraception behavior survey questions to a checklist approach may better engage the participant to elicit more accurate and detailed information. IUD, intrauterine device.

PRAMS and other family planning research initiatives may consider our qualitative themes for possible contraception nonuse barriers. Not only did qualitative inquiry identify additional barriers, but we found respondent frustration with the exclusion of same-sex partners, religious and cultural values, and those struggling with infertility. Our qualitative findings can be used to explore phenomena such as access barriers and the breastfeeding–contraception relationship. We encourage these categories to be more inclusive of gender, sexual orientation, and culture. Further, misreports found in the dichotomized pregnancy prevention question can be eliminated by asking all individuals about each specific contraception behavior and provide a richer understanding of pregnancy prevention.

In conclusion, rigorous standards of both qualitative and quantitative epistemologies can be used as a mixed-method approach to strengthen survey methodology. The depth and breadth of this developed protocol may lead to more accurate findings and resulting health interventions that are more specific to the circumstances surrounding themes identified within participants’ unprompted responses.

Supporting information

S1 Checklist. Standards for Reporting Qualitative Research (SRQR) [18].

(PDF)

Click here for additional data file.^{(159KB, pdf)}

Acknowledgments

We would like to thank the PRAMS Working Group for data collection and review of this article.

Data Availability

The data underlying the results presented in the study are available for request from the CDC PRAMS working group at https://www.cdc.gov/prams/prams-data/researchers.htm.

Funding Statement

The author(s) received no specific funding for this work.

References

1.Tsui A, McDonald-Mosley R, Burke A. Family Planning and the Burden of Unintended Pregnancies. Epidemiol Rev. 2010;32:152–74. doi: 10.1093/epirev/mxq012 [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Sonfield A, Hasstedt K, Kavanaugh ML, Anderson R. The Social and Economic Benefits of Women ‘ s Ability To Determine Whether and When to Have Children. Guttmacher Inst. 2013. [Google Scholar]
3.Bader V, Kelly PJ, Cheng AL, Witt J. The Role of Previous Contraception Education and Moral Judgment in Contraceptive Use. J Midwifery Womens Health. 2014;59. doi: 10.1111/jmwh.12149 [DOI] [PubMed] [Google Scholar]
4.Akers AY, Gold M, a, Borrero S, Santucci A, Schwarz EB. Providers’ perspectives on challenges to contraceptive counseling in primary care settings. J Women’s Health. 2010. doi: 10.1089/jwh.2009.1735 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Richards NK, Crockett E, Morley CP, Levandowski BA. Young women’s reproductive health conversations: Roles of maternal figures and clinical practices. PLoS ONE. 2020;15:e0228142. doi: 10.1371/journal.pone.0228142 [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Kilfoyle KA, Vitko M, O’Conor R, Bailey SC. Health Literacy and Women’s Reproductive Health: A Systematic Review. J Women’s Health. 2016. doi: 10.1089/jwh.2016.5810 [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Davis TC, Fredrickson DD, Potter L, Brouillette R, Bocchini AC, Williams MV, et al. Patient understanding and use of oral contraceptive pills in a southern public health family planning clinic. South Med J. 2006. doi: 10.1097/01.smj.0000223734.77882.b2 [DOI] [PubMed] [Google Scholar]
8.Yee LM, Simon MA. The role of health literacy and numeracy in contraceptive decision-making for urban Chicago women. J Community Health. 2014. doi: 10.1007/s10900-013-9777-7 [DOI] [PubMed] [Google Scholar]
9.Shulman HB, D’Angelo DV, Harrison L, Smith RA, Warner L. The Pregnancy Risk Assessment Monitoring System (PRAMS): Overview of Design and Methodology. Am J Public Health. 2018;108:1305–13. doi: 10.2105/AJPH.2018.304563 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Nettleman MD, Chung H, Brewer J, Ayoola A, Reed PL. Reasons for unprotected intercourse: analysis of the PRAMS survey. Contraception. 2007;75. doi: 10.1016/j.contraception.2007.01.011 [DOI] [PubMed] [Google Scholar]
11.Fowler FJ. Survey Research Methods. 5th ed. Sage Publications, Inc.; 2014. [Google Scholar]
12.Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs—Principles and practices. Health Serv Res. 2013;48. doi: 10.1111/1475-6773.12117 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Center for Disease Control and Prevention. PRAMS Questionnaires. 2018 [cited 2020 May 26]. Available from: https://www.cdc.gov/prams/questionnaire.htm#previous.
14.Mays N, Pope C. Assessing quality in qualitative research. Br Med J. 2000;320:50. doi: 10.1136/bmj.320.7226.50 [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Institute of Medicine (US) Committee on Monitoring Access to Personal Health Care Services. A Model for Monitoring Access. In: Millman M, editor. Access to Health Care in America. National Academics Press; 1993. [PubMed] [Google Scholar]
16.O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341. doi: 10.1136/bmj.c4587 [DOI] [PubMed] [Google Scholar]
17.Cabrera LY, Reiner PB. A Novel Sequential Mixed-method Technique for Contrastive Analysis of Unscripted Qualitative Data: Contrastive Quantitized Content Analysis. Sociol Methods Res. 2016;47:532–48. doi: 10.1177/0049124116661575 [DOI] [Google Scholar]
18.O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: A synthesis of recommendations. Acad Med. 2014;89. doi: 10.1097/ACM.0000000000000388 [DOI] [PubMed] [Google Scholar]
19.Popping R. Analyzing Open-ended Questions by Means of Text Analysis Procedures. BMS Bull Sociol Methodol Bull Methodol Sociol. 2015;128. doi: 10.1177/0759106315597389 [DOI] [Google Scholar]
20.National Center for Health Statistics. Section E: Contraceptive History and Pregnancy Wantedness. National Survey of Family Growth [Internet]. 2017. [cited 24 Jun 2020] pp. 7–9. Available from: https://www.cdc.gov/nchs/data/nsfg/NSFG_PUF2015-2017_FemE_CRQ.pdf. [Google Scholar]

PLoS Med. doi: 10.1371/journal.pmed.1003878.r001

Decision Letter 0

Caitlin Moyer

22 Mar 2021

Dear Dr Richards,

Thank you for submitting your manuscript entitled "Using qualitative inquiry to improve survey data collection: An evaluation of PRAMS postpartum contraception behavior questions" for consideration by PLOS Medicine.

Your manuscript has now been evaluated by the PLOS Medicine editorial staff and I am writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by .

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Caitlin Moyer, Ph.D.

Associate Editor

PLOS Medicine

PLoS Med. doi: 10.1371/journal.pmed.1003878.r002

Decision Letter 1

Caitlin Moyer

23 Jun 2021

Dear Dr. Richards,

Thank you very much for submitting your manuscript "Using qualitative inquiry to improve survey data collection: An evaluation of PRAMS postpartum contraception behavior questions" (PMEDICINE-D-21-00947R1) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to three independent reviewers. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Jul 14 2021 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Caitlin Moyer, Ph.D.

Associate Editor

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

1. Abstract: Please combine the Methods and Findings sections into one section, “Methods and findings”.

2. Abstract: Methods and Findings: Please provide some additional details on the PRAMS survey, including the individuals represented and setting (e.g. the US).

3. Abstract: Methods and Findings: In the last sentence of the Abstract Methods and Findings section, please describe the main limitation(s) of the study's methodology.

4. Abstract: Conclusions: Please address the study implications both for the PRAMS questionnaire and if possible the implications for other similar surveys that may be used to evaluate postpartum pregnancy prevention use or non-use.

5. Author summary: At this stage, we ask that you include a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary

6. As noted by Reviewer 1, the survey data seem to be relatively out of date. Please update with more recent data.

7. Methods: Thank you for including the SRQR checklist. Please add the following statement, or similar, to the Methods: "This study is reported as per the Standards for Reporting Qualitative Research (SRQR) guideline (S1 Checklist)."

8. Methods: Did your study have a prospective protocol or analysis plan? Please state this (either way) early in the Methods section.

a) If a prospective analysis plan (from your funding proposal, IRB or other ethics committee submission, study protocol, or other planning document written before analyzing the data) was used in designing the study, please include the relevant prospectively written document with your revised manuscript as a Supporting Information file to be published alongside your study, and cite it in the Methods section. A legend for this file should be included at the end of your manuscript.

b) If no such document exists, please make sure that the Methods section transparently describes when analyses were planned, and when/why any data-driven changes to analyses took place.

c) In either case, changes in the analysis-- including those made in response to peer review comments-- should be identified as such in the Methods section of the paper, with rationale.

9. Methods: Please include the relevant PRAMS questions as a supporting information file, if possible (or please indicate if these are Core Question 50 and 51 that are provided in Figure 1).

10. Findings: Please rename this section “Results”

11. Results: It may be helpful to provide a table with relevant demographic/characteristics of the study population.

12. Results: Line 118-119: Rather than referring to “numerous respondents” it would be helpful here to provide the number/percentage. Similarly, this could be changed at line 123 where “many respondents” are indicated.

13. Discussion: Please present and organize the Discussion as follows: a short, clear summary of the article's findings; what the study adds to existing research and where and why the results may differ from previous research; strengths and limitations of the study; implications and next steps for research, clinical practice, and/or public policy; one-paragraph conclusion.

14. Line 206-207: Please add the ethical approval information to the Methods section.

15. References: Please use the "Vancouver" style for reference formatting, and see our website for other reference guidelines https://journals.plos.org/plosmedicine/s/submission-guidelines#loc-references

16. Table 1: Please make it clear in the second part of the table, that the denominators for the Non-Use reasons are 6044 for the “Raw Sample” and 6719 for the “Recoded Sample” if accurate. Please consider changing “raw” to “original” or similar, to clarify the meaning.

17. Figure 1: Please spell out the abbreviation NFP in the figure or legend.

18. SRQR Checklist: Please use section and paragraph numbers, instead of page and line numbers, to refer to locations within the manuscript.

Comments from the reviewers:

Reviewer #1:

Although the topic is important and I applaud the authors for examining qualitative data to reduce measurement error and make recommendations to strengthen surveillance in the future, I had some concerns as well as recommendations to improve the paper.

General

* When linking pospartum contraception with outcomes (e.g., UIP), make sure you add "use" - since postpartum contraception must be used to impact outcomes.

* PRAMS data are population-based but not nationally representative (although they represent a large proportion of US births). Please correct language in paper.

* The paper would benefit from editing.

* The data are quite outdated - from 2012. 2019 PRAMS data are now available. The findings would be more helpful if the analysis were re-done using the most updated data available.

* Contraceptive use behaviors are complex and have to be analyzed carefully. This includes data from PRAMS as well as other data systems (e.g., BRFSS). It is quite common for someone to report not using a method, but then listing a method as a reason for not using contraception (e.g. tubal ligation or partner sterilization). This isn't an issue with question reliability; respondents just don't think that something they don't do daily constitutes contraception use. So recoding is always recommended.

* As a PRAMS analyst, I am a proponent of examining the write-in responses and coding as appropriate. But for a publication in the peer-reviewed literature, I am more interested in how such efforts change the prevalence in the behavior - (especially one that is tracked for Healthy People). As such, I wish the authors would apply the recodes of reasons for non-contraception use and apply then to the contraceptive use variable (e.g, those using tubal ligation, vasectomy would be considered contraceptive users).

Abstract

* Change 2nd sentence to reflect "…..postpartum contraceptive use and reasons for non-use" (rather than postpartum contraception non-use and reasons for not preventing pregnancy. Not sure what is meant by "not preventing pregnancy".

* Consider making the 3rd sentence specific to PRAMS. Something like "In addition to pre-determined, closed-ended response options on reasons for not using contraception, respondents could write in a response. This qualitative data can be used….

* I'm not sure about "analysis of the write-in responses validates PRAMS". It does reduce misclassification bias since many response options can be recoded to reflect contraceptive use (e.g., when a woman reports partner vasectomy). So it improves the performance/measurement of the indicator, but to validate, you would need to compare against something else (like electronic medical records).

* PRAMS is a complex survey and weights must be applied to interpret estimates. It would be nice to see how the weighted proportions of non-use differed comparing raw versus recodes responses.

* Typically, one would recode responses of "tubal ligation" as current contraceptive use (rather than non-use, at low risk for pregnancy). When you mention "abstinence" do you mean period abstinence/calendar method (which is a contraceptive method); or not being sexually active?

Introduction

* Page 2, line 52 - I understand the concern about self-report leading to bias, but how does the size of the dataset lead to bias?

* Not following page 2, lines 53-56 - how is this an example of bias? The cited report examined a question on contraception use at time of conception. Please make this clearer, since it differs from the main question of interest for this paper - postpartum contraceptive use.

* Page 2, line 57 - "why women do not prevent a subsequent pregnancy". The PRAMS question examine postpartum contraceptive use. Keep the focus on the behavior at hand. We do not know if they go on to become pregnant again or not.

Methods

* I still think there is value in updating with more recent data. If you wish to look at Phase 7 because it includes more response options, at least examine the most recent Phase 7 data which is 2015.

* Didn't quite follow page 3, lines 70-73.

* Page 3, line 77 - not all reasons for 'not using contraception' are barriers - they are often personal choice. Consider rephrasing to something that more closely matches what was measured (e.g., reasons for not using contraception at time of survey).

Results

* Table 1 - to me, the differences between raw and recoded estimates are not that striking for postpartum contraception use. For reasons for non-use, the proportion reporting 'other barriers' is much lower after recoding, which makes sense; but I would expect the proportions in the recoded sample to be higher compared with the raw sample (due to recoding of write-in other responses), but for most, it is lower. Why is this? Since PRAMS data should be weighted, how did the proportions compare (examining raw vs recoded) once the proportions were weighted.

* Table 2 findings are interesting and illuminating. To put into perspective however, these responses represent a very small proportion of PRAMS respondents (<3%).

* Figure 1 - 4th row of boxes - change "preventing pregnancy" and "not preventing pregnancy" to "using contraception" and "not using contraception". Contraception is not 100% effective, so using contraception does not translate to preventing pregnancy (since pregnancies can still occur).

* Figure 1 - can you explain more what is meant by "does open text align with categorical reason for non-use"? Maybe provide some examples of when text does not align? I would also recommend a last step in Figure 1 - apply reasons for non-contraceptive use that are contraceptive methods (tubal ligation/partner vasectomy) to recode the contraceptive use (yes/no) variable.

* Figure 2, Table 1 - for "abstinence" - it's important to distinguish between 'not having sex' a

* Page 5, lines 126-127 - because these are postpartum women, women may not yet be sexually active. But because sexual activity will likely resume in the near future, it is common practice for women to start a contraceptive method before resuming sexual activity. So I'm not sure how much weight I would give to those types of discrepancies. A good example is a woman who starts an IUD or implant in the hospital before discharge but reports being abstinent early postpartum because she has not yet resumed sexual activity.

Discussion

* Lines 161-162 - although discrepancies were found, it was only a very small proportion (<3%). This sentence seems a bit strong. Same comment to page 8, line 191. Can you really say "many" - from my count it was 873/31,208 or 2.8%. Perhaps I am calculating incorrectly?

* Line 164 - change "preventing pregnancy" to "using contraception" - this should be made throughout. Again, per my comment above, this is not that surprising and even expected given the population is postpartum. If examining BRFSS data, then you'd have a stronger case.

* Line 168-169 - although I appreciate the shout out to NSFG which is an amazing data source, I'm not sure how helpful these concluding remarks/conclusions are. NSFG is a survey largely focused on fertility (so more attention can be given to contraceptive use). It is also administered in person so quality of data is higher. PRAMS is a general survey on maternal behaviors and experiences, and postpartum contraceptive use is only a very small component. It's mode of data collection is different than NSFG.

Reviewer #2: This is a well written descriptive study on an important topic. The manuscript will no doubt help researchers in accurately measuring contraceptive prevalence, taking into account the reasons stated for contraceptive non-use.

I have a few points to note. First is; should vasectomy, tubal ligation, abstinence not be considered as contraceptive methods? If the answer is yes, I suppose that contraceptive prevalence is underestimated in PRAMS. That takes me to my second point, I have always believed that contraceptive prevalence is underestimated in large surveys. This manuscript further supports my observation.

I think authors should reflect on how the lack of proper recoding and cleaning of data may result in underestimation of contraceptive prevalence in their discussion.

Reviewer #3: This brief manuscript investigates the effects of interpreting and back-coding open-ended survey responses on population estimates from the PRAMS Pregnancy Risk Monitoring System. For each variable code examined, this exercise altered response distributions, although, thankfully, not to an appreciable extent. Nonetheless, the analyses presented reflect improved estimates and might suggest to the PRAMS data management team that they consider routine application of these or similar back-coding practices when finalizing public use versions of PRAMS data.

The analyses also generated recommendations for revision to the block of PRAMS questions that ask about reasons for not engaging in postpartum contraception. The most important of these recommendations is that the main item ("Are you or your partner doing any of the following to keep from getting pregnant?") be revised from a Check-all-that-apply format to one that asks respondents to answer "yes" or "no" for each response option. Structuring these questions as recommended by these authors is in fact a piece of convention wisdom in the survey methodology literature and it is somewhat surprising that PRAMS is not already doing so. Hopefully, this research will encourage them to consider doing so. One suggestion, however, in the author's recommended reformatting, which is presented in Figure 2, is to remove the "Check all that apply" instruction from the question stem, as that ironically harkens back to the previous format, which they are recommending be revised. Indeed, the "check all that apply" instruction may encourage some respondents not to follow the yes-no response format, so I strongly recommend it be removed here. Otherwise, this is a well written and useful research note, one that employs statistical methods appropriate for the data and research questions. I believe it has the potential to make a good contribution to the PRAMS research process.

Any attachments provided with reviews can be seen via the following link:

[LINK]

PLoS Med. 2022 Jan 5;19(1):e1003878. doi: 10.1371/journal.pmed.1003878.r003

Author response to Decision Letter 1

20 Jul 2021

Attachment

Submitted filename: Response to Reviewers PLOSMed.docx

Click here for additional data file.^{(20.6KB, docx)}

PLoS Med. doi: 10.1371/journal.pmed.1003878.r004

Decision Letter 2

Caitlin Moyer

26 Aug 2021

Dear Dr. Richards,

Your revised paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent back to one of the reviewers. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of the reviewer feedback, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

We expect to receive your revised manuscript by Sep 16 2021 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

We look forward to receiving your revised manuscript.

Sincerely,

Caitlin Moyer, Ph.D.

Associate Editor

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

1. Please fully address the remaining points mentioned by Reviewer 1.

2. Title: Please revise your title according to PLOS Medicine's style. Your title must be nondeclarative and not a question. It should begin with main concept if possible. "Effect of" should be used only if causality can be inferred, i.e., for an RCT. Please place the study design ("A randomized controlled trial," "A retrospective study," "A modelling study," etc.) in the subtitle (ie, after a colon).

3. Abstract: Line 20-21: We suggest removing “indicating” and replacing with “potentially indicating” or “suggesting” as this is an interpretation.

4. Abstract: conclusions: Line 24-25: We suggest noting “Respondents reporting contraception non-use often described in open-text responses other methods of pregnancy prevention…”

At Line 27-28, we suggest clarifying this sentence: “Our findings contribute to the understanding of how open-text survey questions can lead to more accurate depiction of postpartum contraceptive behavior.” or similar.

5. The Author Summary should be formatted into 3 sections (Why was this study done; What did the researchers do and find; What do these findings mean), with 2-3 bulleted points per section. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary

6. Introduction: The objectives presented in the final paragraph could be presented more clearly. It would be helpful if the introduction went into more detail on the significance of the study objective (such as potential for misinterpretation of questions/response options, reasons falling beyond categorical options presented; and how open-text response data may be informative in combination with quantitative data).

7. Methods: Line 76. Please report the number of individuals included in the PRAMS survey/included in this study. Please fully describe the population and setting. Please note whether all postpartum women were mailed the questionnaires, and provide details on how a sample was selected from birth certificate data for inclusion in the PRAMS. Please note survey response rates. Please note early on whether those included are nationally representative of the population of post-partum women in the United States.

8. Methods: Line 82: Please note the year(s) of the Phase 7 PRAMS.

9. Methods: Line 84-89: Please provide more description of the background for this process, keeping in mind this would be intended for a broad clinical/medical audience.

10. Methods: Original data analysis: Lines 91-103: Please mention here where the open-text responses are incorporated/evaluated, including the numbers of qualitative responses.

11. Methods: Line 112: Please note if the “PRAMS examples” indicates the categorical response options provided.

12. Methods: Please note in the Methods if your study had a prospectively developed analysis plan.

Did your study have a prospective protocol or analysis plan? Please state this (either way) early in the Methods section.

b) If no such document exists, please make sure that the Methods section transparently describes when analyses were planned, and when/why any data-driven changes to analyses took place.

c) In either case, changes in the analysis-- including those made in response to peer review comments-- should be identified as such in the Methods section of the paper, with rationale.

13. Methods: Qualitative analysis: Please completely describe the conceptual framework underlying your qualitative analysis.It would be helpful to provide more description on this process, keeping in mind a broad, general audience.

14. Results: Line 133: It is slightly confusing to refer to this as “original sample” and could be renamed “original response data” or similar.

15. Results: Please mention the total numbers included in the sample. Please present numerators and denominators for percentages.

16. Results: Line 140: Please clarify if the 30% reporting additional reasons indicates open-text responses.

17. Results: Line 153: Please provide the number of responses that were recoded based on the open-text responses.

18. Discussion: The organization of the discussion could be further refined. Please present and organize the Discussion as follows: a short, clear summary of the article's findings; what the study adds to existing research and where and why the results may differ from previous research; strengths and limitations of the study; implications and next steps for research, clinical practice, and/or public policy; one-paragraph conclusion.

Please begin with a statement reflecting the main findings of the study, rather than beginning with interpretations (“...illuminated an unclear understanding…”). We suggest expanding on the discussion of existing literature, and limitations of the study.

19. Line 186: Please clarify the sentence describing “respondents missing skip patterns”

20. Line 236: Please move this statement to the Methods, including the exemption number.

21. Line 238: Please remove the Consent for Publication statement.

22. Line 239: Please remove this section, and ensure all information on data availability are included in the manuscript submission metadata.

23. Lines 242-249: Please remove the Competing Interests, Funding, and Author contributions sections from the main text, and check that this information is complete and accurate in the relevant sections of the manuscript submission metadata.

24. Table 2: Please include the definition of all abbreviations in the legend (LAM).

25. Figure 2: Please define the abbreviation SP.

Comments from the reviewers:

Reviewer #1: * PRAMS data are population-based but not nationally representative (although they represent a large proportion of US births). The text (e.g., abstract) still refers to PRAMS as being nationally representative which is incorrect.

* PRAMS Phase 7 was conducted 2012-2015; PRAMS Phase 8 began in 2016 and 2019 data are now publicly available with 2020 data expected in the coming months. I can understand why the authors do not wish to replicate the work using Phase 8 2019 data given this would be very time intensive, but without doing so, I'm not sure how informative the report is related to the last sentence of the abstract: "This PRAMS question can be revised to improve clarity, including a more comprehensive list of pregnancy prevention methods and non-use options." Perhaps drop that sentence and make the conclusions a bit more broad - something along the lines of how examining write-in response can improve survey quality to inform both PRAMS and other similar surveys.

* I do not agree with the author's response to not present weighted estimates of contraceptive non-use. Estimates of PRAMS data should always be weighted so the estimates are population-based. Of course the in-depth examination of write-in responses is done at the individual level, but for one to meaningfully compare how different or similar estimates of non-use are comparing original/raw vs recoded data, those estimates should be weighted. Otherwise the reader cannot tell if the recoding had a meaningful impact on proportions.

New sentence added to the discussion in response to Reviewer #2 seems a bit off to the reviewer's intent. Rather than "A failure to capture contraception behaviors and methods like abstinence and sterilization may contribute to an underestimate of those preventing pregnancy"….I suggest framing it around the importance of data cleaning/recoding and examining write-in response to improve data quality/validity. Because in reality, PRAMS will not be able to add new response options. To add a new response option, another one must come off given space limitations on the survey (PRAMS is hard copy vs BRFSS which is phone-administered and not every response is read by the interviewer). PRAMS questions deemed to have too many possible response options to be valid might instead be removed from the survey altogether, which I believe would be an unintended consequence of the paper. So careful framing is needed. The data aren't junk - analysts should just carefully examine write-in responses and recode appropriately.

Any attachments provided with reviews can be seen via the following link:

[LINK]

PLoS Med. 2022 Jan 5;19(1):e1003878. doi: 10.1371/journal.pmed.1003878.r005

Author response to Decision Letter 2

16 Sep 2021

Attachment

Submitted filename: 2Response to Reviewers PLOSMed.docx

Click here for additional data file.^{(21.8KB, docx)}

PLoS Med. doi: 10.1371/journal.pmed.1003878.r006

Decision Letter 3

Caitlin Moyer

12 Nov 2021

Dear Dr. Richards,

Thank you very much for re-submitting your manuscript "Improving pregnancy prevention behavior data collection: a mixed-method inquiry of PRAMS postpartum contraception questions" (PMEDICINE-D-21-00947R3) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor and it was also seen again by one of the reviewers. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We expect to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

Please note, when your manuscript is accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you've already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosmedicine@plos.org.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.

We look forward to receiving the revised manuscript by Nov 19 2021 11:59PM.

Sincerely,

Caitlin Moyer, Ph.D.

Associate Editor

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

1. Please address the points of Reviewer 1, including avoiding referring to the PRAMS survey as national (e.g at line 4, 67, and throughout) and tempering conclusions to reflect that original and re-coded findings lead to similar estimates from a clinical perspective, but noting that examination of responses could be useful for improving data quality.

2. Title: We suggest revising to "Use of open-text responses to recode categorical survey data on postpartum contraception use among women in the United States: A mixed-methods inquiry of Pregnancy Risk Assessment Monitoring System data" or similar, but please include "United States" in the title.

3. Abstract: Line 4: Please change this to “Pregnancy Risk Assessment Monitoring System (PRAMS).

4. Abstract: Line 24-25: We suggest revising the sentence to: “In this study, we observed that respondents reporting contraception non-use…” or similar.

5. Author Summary: Please organize the author summary into sets of 2-3 single sentence bullet points for each of the following questions. For an example, please see: https://doi.org/10.1371/journal.pmed.1002416

Descriptions of each of the three questions are included below. For example, the first two points could fall under the first question, the third, fourth, and fifth points could go under the second question, and the sixth and seventh points could go under the third question.

-Why Was This Study Done? Authors should reflect on what was known about the topic before the research was published and why the research was needed.

-What Did the Researchers Do and Find? Authors should briefly describe the study design that was used and the study’s major findings. Do include the headline numbers from the study, such as the sample size and key findings.

-What Do These Findings Mean? Authors should reflect on the new knowledge generated by the research and the implications for practice, research, policy, or public health. Authors should also consider how the interpretation of the study’s findings may be affected by the study limitations.

6. Author summary: Please revise the following bullet points:

-”We used qualitative methods to uncover richer information about respondents’ postpartum contraception behavior and interpretations of the survey question” -please remove “richer” or replace with “more detailed information” or similar.

-We suggest revising the next point to: “We observed that qualitative responses added to and altered quantitative contraception behavior reports, and that reasons for contraception non-use are incompletely captured by existing response categories.” or similar.

-We suggest revising this point to: “Family planning professionals may be able to obtain significant and detailed information about contraceptive behavior from open-ended responses, and this information may be useful to adjust clinical approaches to postpartum contraception conversations.” or similar.

7. Introduction: Line 67-68: Please refer to PRAMS as a population-based rather than national surveillance system.

8. Introduction: Line 75-77: Please revise to: “The objective of these analyses was to identify how responses to PRAMS survey questions match respondents’ consistent and accurate (reliability and validity) interpretations through the additional analysis of qualitative observations.” or similar, to clarify.

9. Methods: Line 79: Please revise the following to provide more relevant information on the PRAMS project (https://www.cdc.gov/prams/index.htm):

“PRAMS, a surveillance project run by the Centers for Disease Control and Prevention (CDC) and state and local health departments, collects state birth certificate data and includes both universal core questions and site-specific questions about maternal attitudes and behaviors related to pregnancy [reference].” or similar.

Please mention the fact that PRAMS covers approximately 81% of births in the United States (or please cite the most accurate estimate from the CDC).

10. Methods: Line 80: Please provide more details on how postpartum women were selected to be mailed the questionnaires (e.g. it seems as if questionnaires are sent 2-4 months after delivery, to a sample drawn monthly from current birth certificate data).

11. Methods: Line 86: In the abstract, PRAMS 2012 data are mentioned, and here a range of 2012-2015 is mentioned. Please clarify the year during which survey data were collected for this analysis.

12. Methods: Line 110: Please check if this should be “...postpartum contraception uptake barriers…”

13. Methods: Line 117: We suggest noting that “Original postpartum contraception use prevalence data were analyzed using SPSS software version 26 (IBM).” or similar. Please similarly clarify this for the mention of Excel (Line 119), Stata (Line 145) and NVivo (Line 148).

14. Results: Line 181: Please remove “...suggesting the importance of recoding.” as this is an interpretation.

15. Discussion: As Reviewer 1 points out, please temper conclusions to mention and reflect on the fact that the original and recoded findings are significantly different statistically, but are similar, in terms of clinical implications but that the examination of responses could be useful for improving data quality.

16. Discussion: Line 215: We suggest revising to: “...this suggests that additional categorical options could be useful to capture a more comprehensive picture of contraception behavior.”

17. Discussion: Line 218: We suggest removing the word “national” and replacing with “population-based” or similar.

18. Discussion: Line 234: Please avoid the word “national” and replace with “population-based” or similar.

19. Discussion: Line 260: Please revise to “may lead to” in the sentence.

20. References: Please provide the complete citation information for ref 14.

21. Table 1: Please do not report "p<0.05" and instead provide exact p values for each analysis in the weighted % column (including p values for results not reaching statistical significance).

22. Figure 1: There seem to be two versions of this figure. Please include a descriptive legend in addition to the title, for this figure. Please indicate natural family planning (NFP) in the legend.

23. Figure 2: Please include a descriptive legend in addition to the title for this figure. Please define all abbreviations used in the legend (IUD).

24. SRQR Checklist: Please refer to Methods, paragraph 2 as the location for IRB/ethical review information. Please also refer to Competing Interests and Financial Disclosures sections as locations for Conflict of Interest and Funding information.

Comments from Reviewers:

Reviewer #1: PRAMS data are population-based but not national (although they represent a large proportion of US births); data are state/site-specific not national. Abstract line 4 still refers to PRAMS as being national.

I appreciate the authors comparing weighted proportions of original vs recoded variables; however, although statistically different via chi-square tests (page 8, lines 208-209), the differences are not meaningful from a clinical or public health surveillance perspective. When I look at Table 1 - I feel reassured that both methods (original vs recoded) produce very similar estimates of postpartum contraception use (79% vs 77%). That's an OK conclusion to have as well; you could still encourage analysts to examine write-ins (time permitting) to improve data quality.

Any attachments provided with reviews can be seen via the following link:

[LINK]

PLoS Med. 2022 Jan 5;19(1):e1003878. doi: 10.1371/journal.pmed.1003878.r007

Author response to Decision Letter 3

19 Nov 2021

Attachment

Submitted filename: 3Author Response 111621.docx

Click here for additional data file.^{(18KB, docx)}

PLoS Med. doi: 10.1371/journal.pmed.1003878.r008

Decision Letter 4

Caitlin Moyer

29 Nov 2021

Dear Dr Richards,

On behalf of my colleagues and the Academic Editor, Sarah Stock, I am pleased to inform you that we have agreed to publish your manuscript "Use of open-text responses to recode categorical survey data on postpartum contraception use among women in the United States: A mixed-methods inquiry of Pregnancy Risk Assessment Monitoring System data" (PMEDICINE-D-21-00947R4) in PLOS Medicine.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

We also request that you address the following editorial issues:

1. Author summary: Please revise and re-organize the points of the Author Summary into three sections as follows:

Why was this study done?

-Survey respondents can occasionally provide open-ended answers that may be overlooked during quantitative data analysis

- This study was conducted to study how qualitative inquiry of open-text responses to a survey question related to postpartum contraception use can be used to provide insight into data accuracy within quantitative surveys.

What did the researchers do and find?

-Using a mixed-method approach, we calculated descriptive statistics from PRAMS survey responses from 31,208 postpartum women in the US who provided categorized reasons for not using contraception and qualitatively analyzed respondents’ adjacent open-ended written answers to uncover more detailed information about respondents’ postpartum contraception behavior and interpretations of the survey question.

-Through systematic recategorization, we found that qualitative responses added to and altered quantitative contraception behavior reports, and that reasons for contraception non-use are incompletely captured by existing response categories.

What do these findings mean?

-These findings suggest that researchers may be able to gain a better understanding of postpartum pregnancy prevention by using mixed-method analyses on categorical and open-ended responses.

-While qualitative inquiry may yield subjective interpretation, rigorous pragmatic data management can strengthen survey questions through open-text responses not captured in existing quantitative data.

-Family planning professionals may be able to obtain significant and detailed information about contraceptive behavior from open-ended responses, and this information may be useful to adjust clinical approaches to postpartum contraception conversations.

2. Methods: Line 148: Please change this to “mixed-method” lens.

3. Discussion: Lines 254-258: Please revise to: “The statistically significant differences suggest the importance of re-examining survey responses to improve data quality; however, it should be noted that the original and recoded findings yield similar measures of contraception use. Overall, these PRAMS questions illuminated an unclear understanding of the definition of pregnancy prevention in some instances, and this may have contributed to inconsistent answers based on respondent interpretation.”

4. Discussion: Line 266: Please remove the word “rich” from the sentence.

5. Discussion: At line 268, and line 272, please avoid the use of italics for emphasis.

6. Table 1: Please report p values as p<0.001 for each comparison where applicable, rather than p=0.000. Please indicate if the comparison of original and recoded responses for who report “No” to the question on use of contraception method was tested (it is not indicated as p<0.001).

7. Figure 1 and Figure 2: Thank you for providing the legends for the figures. Please note that the figures themselves are no longer included with this version of the submission. These were previously included as files named “Survey Figure 1” and “Survey Figure 2”. Please provide and re-incorporate these into the manuscript.

In the meantime, please log into Editorial Manager at http://www.editorialmanager.com/pmedicine/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.

PRESS

We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with medicinepress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

Thank you again for submitting to PLOS Medicine. We look forward to publishing your paper.

Sincerely,

Caitlin Moyer, Ph.D.

Associate Editor

PLOS Medicine

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Checklist. Standards for Reporting Qualitative Research (SRQR) [18].

(PDF)

Click here for additional data file.^{(159KB, pdf)}

Attachment

Submitted filename: Response to Reviewers PLOSMed.docx

Click here for additional data file.^{(20.6KB, docx)}

Attachment

Submitted filename: 2Response to Reviewers PLOSMed.docx

Click here for additional data file.^{(21.8KB, docx)}

Attachment

Submitted filename: 3Author Response 111621.docx

Click here for additional data file.^{(18KB, docx)}

Data Availability Statement

The data underlying the results presented in the study are available for request from the CDC PRAMS working group at https://www.cdc.gov/prams/prams-data/researchers.htm.

[pmed.1003878.ref001] 1.Tsui A, McDonald-Mosley R, Burke A. Family Planning and the Burden of Unintended Pregnancies. Epidemiol Rev. 2010;32:152–74. doi: 10.1093/epirev/mxq012 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pmed.1003878.ref002] 2.Sonfield A, Hasstedt K, Kavanaugh ML, Anderson R. The Social and Economic Benefits of Women ‘ s Ability To Determine Whether and When to Have Children. Guttmacher Inst. 2013. [Google Scholar]

[pmed.1003878.ref003] 3.Bader V, Kelly PJ, Cheng AL, Witt J. The Role of Previous Contraception Education and Moral Judgment in Contraceptive Use. J Midwifery Womens Health. 2014;59. doi: 10.1111/jmwh.12149 [DOI] [PubMed] [Google Scholar]

[pmed.1003878.ref004] 4.Akers AY, Gold M, a, Borrero S, Santucci A, Schwarz EB. Providers’ perspectives on challenges to contraceptive counseling in primary care settings. J Women’s Health. 2010. doi: 10.1089/jwh.2009.1735 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pmed.1003878.ref005] 5.Richards NK, Crockett E, Morley CP, Levandowski BA. Young women’s reproductive health conversations: Roles of maternal figures and clinical practices. PLoS ONE. 2020;15:e0228142. doi: 10.1371/journal.pone.0228142 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pmed.1003878.ref006] 6.Kilfoyle KA, Vitko M, O’Conor R, Bailey SC. Health Literacy and Women’s Reproductive Health: A Systematic Review. J Women’s Health. 2016. doi: 10.1089/jwh.2016.5810 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pmed.1003878.ref007] 7.Davis TC, Fredrickson DD, Potter L, Brouillette R, Bocchini AC, Williams MV, et al. Patient understanding and use of oral contraceptive pills in a southern public health family planning clinic. South Med J. 2006. doi: 10.1097/01.smj.0000223734.77882.b2 [DOI] [PubMed] [Google Scholar]

[pmed.1003878.ref008] 8.Yee LM, Simon MA. The role of health literacy and numeracy in contraceptive decision-making for urban Chicago women. J Community Health. 2014. doi: 10.1007/s10900-013-9777-7 [DOI] [PubMed] [Google Scholar]

[pmed.1003878.ref009] 9.Shulman HB, D’Angelo DV, Harrison L, Smith RA, Warner L. The Pregnancy Risk Assessment Monitoring System (PRAMS): Overview of Design and Methodology. Am J Public Health. 2018;108:1305–13. doi: 10.2105/AJPH.2018.304563 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pmed.1003878.ref010] 10.Nettleman MD, Chung H, Brewer J, Ayoola A, Reed PL. Reasons for unprotected intercourse: analysis of the PRAMS survey. Contraception. 2007;75. doi: 10.1016/j.contraception.2007.01.011 [DOI] [PubMed] [Google Scholar]

[pmed.1003878.ref011] 11.Fowler FJ. Survey Research Methods. 5th ed. Sage Publications, Inc.; 2014. [Google Scholar]

[pmed.1003878.ref012] 12.Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs—Principles and practices. Health Serv Res. 2013;48. doi: 10.1111/1475-6773.12117 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pmed.1003878.ref013] 13.Center for Disease Control and Prevention. PRAMS Questionnaires. 2018 [cited 2020 May 26]. Available from: https://www.cdc.gov/prams/questionnaire.htm#previous.

[pmed.1003878.ref014] 14.Mays N, Pope C. Assessing quality in qualitative research. Br Med J. 2000;320:50. doi: 10.1136/bmj.320.7226.50 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pmed.1003878.ref015] 15.Institute of Medicine (US) Committee on Monitoring Access to Personal Health Care Services. A Model for Monitoring Access. In: Millman M, editor. Access to Health Care in America. National Academics Press; 1993. [PubMed] [Google Scholar]

[pmed.1003878.ref016] 16.O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341. doi: 10.1136/bmj.c4587 [DOI] [PubMed] [Google Scholar]

[pmed.1003878.ref017] 17.Cabrera LY, Reiner PB. A Novel Sequential Mixed-method Technique for Contrastive Analysis of Unscripted Qualitative Data: Contrastive Quantitized Content Analysis. Sociol Methods Res. 2016;47:532–48. doi: 10.1177/0049124116661575 [DOI] [Google Scholar]

[pmed.1003878.ref018] 18.O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: A synthesis of recommendations. Acad Med. 2014;89. doi: 10.1097/ACM.0000000000000388 [DOI] [PubMed] [Google Scholar]

[pmed.1003878.ref019] 19.Popping R. Analyzing Open-ended Questions by Means of Text Analysis Procedures. BMS Bull Sociol Methodol Bull Methodol Sociol. 2015;128. doi: 10.1177/0759106315597389 [DOI] [Google Scholar]

[pmed.1003878.ref020] 20.National Center for Health Statistics. Section E: Contraceptive History and Pregnancy Wantedness. National Survey of Family Growth [Internet]. 2017. [cited 24 Jun 2020] pp. 7–9. Available from: https://www.cdc.gov/nchs/data/nsfg/NSFG_PUF2015-2017_FemE_CRQ.pdf. [Google Scholar]

PERMALINK

Use of open-text responses to recode categorical survey data on postpartum contraception use among women in the United States: A mixed-methods inquiry of Pregnancy Risk Assessment Monitoring System data

Nicole K Richards

Christopher P Morley

Martha A Wojtowycz

Erin Bevec

Brooke A Levandowski

Roles

Abstract

Background

Methods and findings

Conclusions

Author summary

Why was this study done?

What did the researchers do and find?

What do these findings mean?

Introduction

Methods

Fig 1. Steps for qualitative recoding PRAMS Phase VII core question 50 and 51 via open-text responses.

Original data analysis

Recoding variables

Qualitative analysis

Results

Original response data

Table 1. Original and recoded postpartum contraception behavior (n = 31,208) and reasons for nonuse among PRAMS Phase VII (2012) respondents.

Recoded data

Open-text qualitative themes

Table 2. Qualitative themes of postpartum contraception nonuse reasons among PRAMS Phase VII (2012) respondents.

Discussion

Fig 2. Example of comprehensive contraception behavior survey questions.

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Caitlin Moyer

Roles

Decision Letter 1

Caitlin Moyer

Roles

Author response to Decision Letter 1

Decision Letter 2

Caitlin Moyer

Roles

Author response to Decision Letter 2

Decision Letter 3

Caitlin Moyer

Roles

Author response to Decision Letter 3

Decision Letter 4

Caitlin Moyer

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases