Using Data Science to Examine Conflict-Related Sexual Violence

Abstract

The lack of data is a major barrier to comprehending conflict-related sexual violence (CRSV). Existing studies indicate a high prevalence of violence but limited documentation. Media are often the first to report CRSV, thus, can data science methods analyze news reports promptly and accurately? We compare manual, machine-learning and Generative AI analysis of thousands of media reports. Five variables can be automatically coded with high accuracy, increasing to over 90% with “chain of thought” prompting. Data science can reveal previously unknown CRSV attributes to inform timely prevention, but it has important limitations that researchers, advocates, and policymakers should be aware of.

Fifteen years of the UN mandate to collect data on conflict-related sexual violence (CRSV), ¹ make it an important moment to study reports of this violence across diverse conflict situations globally. Silence and the lack of official reporting of CRSV, however, represent major barriers to understanding of this violence as well as the impunity for it (Sutrich, 2020). Greater knowledge about CRSV and its patterns is vital for improving responses to it as well as improving decision-making to prevent the incidence of CRSV.

Collecting data on sensitive types of violence on the ground in politicized conflict contexts is frequently difficult. Studies of sexual and gender-based violence point to a “tip of the iceberg” phenomenon, where a high level of prevalence of violence is associated with a low level of reporting in conflict and fragile situations (Palermo et al., 2014). Scholars have argued that CRSV reporting itself can be a politicized practice (Fujii, 2010; Hoover Green, 2017). Official reports of individual incidents may not show the magnitude or the dynamics of CRSV or the intensity and diversity of crimes. Based on what we know about CRSV in past conflicts, it has been important to study what violence is happening across the phases and types of conflict where there is no “real-time” or divergent data to inform prevention (Campbell, 2018). Scholars have explained when and why state and non-state armed groups use CRSV to exert control over territory or expand governance into new territory (Asal & Nagel, 2021). They have shown how CRSV perpetrators target specific social groups and institutions consistent with their ideology (Revkin & Wood, 2021) and used CRSV as a cost-effective mechanism of displacement and accumulation of wealth (i.e., bride price) (Ellsberg et al., 2021). Scholars have also documented how sexual violence perpetrated before war or conflict to coerce marriage, control and punish women, for instance, has contributed to its high prevalence in wartime CRSV (Alexandre & Mutondo, 2021; Davies & True, 2024). There are also comparative and case study accounts that offer rich analyses of the types of CRSV and groups affected (Boesten, 2017; Hoover Green & Cohen, 2021).

These studies have informed different types of knowledge about CRSV, including quantitative trend analyses based on aggregate reports of sexual and gender-based violence, but our knowledge has a time lag of up to a year, depending on the reporting source (Nordås & Cohen, 2021). Data exclusion, bias and immediacy can limit our knowledge about the opportunity to prevent and protect groups from violence earlier. Could AI machine learning or Generative GenAI analysis of media reports provide a “real-time” source of knowledge about CRSV and its patterns?

New studies are emerging that combine social and data science methods to mine unstructured textual data using AI for real-world impact. Examples include Blake et al.'s (2021) use of machine learning (ML) to study geocoded misogynistic tweets and their relationship to a higher incidence of gender-based violent crime, Battisti et al.'s (2024) analysis of anti-gender-based violence tweets by activists, and van Breen et al.'s (2024) usage of violence in public discourse in the Netherlands. Ebner et al. (2024) have combined natural language processing methods with ML to automate the detection of language used by extremists to develop a violent risk index. In this article, we explore the potential for news media reports to be an alternative source of information to prevent and respond to CRSV in different geographic locations. While real-time media may vary, it is generally more up to date with less time lag than official reports between the CRSV event and our knowledge of it.

To maximize the information that we can use to learn about patterns of CRSV across different types of conflict, we collected thousands of media reports and compared their analysis using manual coding, machine learning (ML) NLP models, and GenAI prompting techniques with large language models (LLMs) such as OpenAI's most advanced publicly available model by subscription, GPT-4o released in 2024 (Achiam et al., 2023). GenAI refers to AI models that create new, original content, such as text. Recent advances in GenAI, especially the emergence of LLMs, provide an opportunity to solve language tasks without the need for task-specific datasets and training. LLMs can be harnessed to automatically extract structured knowledge from media reports, such as through information extraction. In this study, we experimented with strategies to leverage LLMs for training-free automated information extraction of key CRSV variables, such as the site of the violence and victim gender. Our aim was to enhance the analysis of some key CRSV patterns. While we conducted this analysis using English language news reports from Factiva (Dow Jones), using the same process, it can be generalized to other languages by adapting the search terms to reflect local language use and further refining the GenAI prompts. Factiva supports content in 32 languages, providing access to over 30,000 sources from 200 countries, and news stories are often published by various sources in multiple languages. Furthermore, in situations where local media reports may be limited and/or published via international outlets because of danger and harm in local contexts with lower press freedom, internet, and repression—international media reports can be an important source.

This article explains the experience of our study to assess the suitability of these methods for the emerging field of CRSV. The article has three parts. First, we review the evolution of the empirical study of CRSV and some of the methodological issues that have arisen as scholars have grappled with how to build knowledge on the immediate causes and patterns of this violence. Second, in four phases, we describe our pilot study to explore the efficacy of using machine learning (ML) and Gen AI data science to build knowledge on CRSV, addressing both the absence of—as well as the gaps in—official reports. We discuss the data preparation, extraction and results for both ML and GenAI pilots. We also explain the validation process that we undertook to test the accuracy of the AI data extraction and coding, and to compare its accuracy with manual coding. Third, and finally, we analyze the methodological challenges and limitations of ML and GenAI tools and how we tried to address them to demonstrate the potential for monitoring and analysis of SGBV in conflict contained in news reports. We conclude by considering the potential of ML and GenAI to improve policy and decision-making and to help us overcome gendered data and methodological bias in the study of CRSV by mining the most up-to-date, data for impact.

Approaches to Documenting CRSV

Foundational research over the past decade has led to the empirical study of CRSV. Cohen and Nordås (2014), for instance, created the first global dataset that facilitates analysis of CRSV patterns. The Sexual Violence in Armed Conflict dataset codes four categories of “sexual violence” recorded in US State Department, Amnesty International and Human Rights Watch reports from conflict-affected countries over a 30 + year period (1989–2021) (Nordås & Cohen, 2021). Dumaine et al. (2022) further extended this data collection in the Repertoires of Sexual Violence in Armed Conflict dataset to cover eight types of sexual violence occurring in conflicts between 1989 and 2015 together with qualitative notes on reported incidents, making it possible to analyze global patterns of CRSV, including patterns of underreporting. The not-for-profit media organization, Women Under Siege (2016), established an online database of testimonials from women and men victims across historical and contemporary conflicts. The Office of the UN Special Representative for the Secretary General on CRSV has produced an annual report since 2012, documenting countries and perpetrators of CRSV recorded in the previous year (UN SRSG, n.d.). This collection of quantitative and qualitative datasets is important for understanding patterns of SGBV, but there are limitations to these datasets, namely reporting exclusions, bias, and immediacy.

First, reporting exclusions. There is the risk that some types of violence and victims of violence are not recorded in the reports relied upon to inform datasets. Experiences of undocumented violence that may be critical to understand CRSV patterns are frequently lost over time (Boesten & Henry, 2018). In addition, a lack of data for some countries or time periods may indicate an oppressive political context that makes it difficult and potentially life-threatening to report sexual and gender-based violence. In low-intensity conflicts such as the Philippines, for example, patterns of sexual violence may be present but not reported or not documented as “conflict-related” (Swaine, 2015, p. 774). Some political or ethnic groups, but not others, may have strong civil society networks that can document and report these experiences to official sources such as the United Nations (Oo & Davies, 2021). Some reports may exclude male victims. For example, du Toit and le Roux (2020) contend that there is a persistent bias against documenting male victims of SGBV in conflict. Touquet et al. (2020) have found that myths and misconceptions of male victims in CRSV leads to limited access to health services, which leads to poor reporting and documentation.

Sexual violence is under-reported everywhere in the world, but reliable documentation of reports of sexual violence during conflict is especially scarce in areas where demographic and health surveys are comparatively infrequent and where national reporting systems are relatively underdeveloped (Kiakuvue et al., 2024). The scarcity of institutions to report to or their inaccessibility in conflict, displacement and rural settings affect our knowledge of the patterns of violence and may lead to extremely low conviction rates for sexual violence, further undermining the incentive to report (Koos & Lindsey, 2022; Willis, 2021). Many countries have no baseline data on any form of sexual and gender-based violence let alone annual documentation of this violence in conflict or emergencies. Poor protection for victims due to ineffective or gender-biased law enforcement and justice systems, and institutional incapacity to record and analyze police or judicial data are also major issues (Haglund & Richards, 2018). Given that social context and knowledge can change over time depending on the course of a conflict, official and retrospective reporting data to study the patterns of CRSV might be misleading if we are attempting to understand current scale and intensity.

The second problem is unintended reporting bias. Reporting trends tend to reflect the dominant crimes, perpetrators and victims, and relationships highlighted in the global discourse on CRSV to date (Baaz & Stern, 2018). Some choices need to be made to manage the sources of reporting data, variables recorded from reports, and the validation process. Datasets are labor-intensive and time-consuming to build, especially if they are reliant on manual coding. All those who code reports, sometimes hundreds per year for a conflict, need to consistently code to ensure internal and inter-coder reliability. Manual coding often requires extensive training, making it a challenge for new scholars to adopt the technique and expand the field (see Fujiwara et al., 2021). Moreover, the integrity of the coding is subject to human fallibility, especially when done at scale. Even with concerted efforts to gather better data through surveys or incident reporting in conflict situations, these methodological dilemmas compound the under-reporting of SGBV. As Hoover Green and Cohen (2021) note, bias, accuracy, and errors can slip into official reports. Moreover, the need for validation leads to, by necessity, choices about what data is collected and what data is excluded (Hoover Green, 2017; Wood, 2018). Desk-based and field-based research may be selective, incomplete, and biased without “methodological innovations—together with a disciplined awareness of the breadth and variety of gendered conflict experiences” (Hoover Green, 2017, p. 325). Improving the tools for data collection and analysis cannot be done in isolation from examining efforts to identify additional sources of CRSV reporting. Official reports, from governments, the United Nations, and humanitarian and international rights agencies, should be viewed as valid and reliable sources to report on the patterns of violence. But there is an unmet need for immediacy and for knowledge of situations where official CRSV reports are necessarily limited.

In conflict-affected contexts with limited awareness, humanitarian presence, technology, and few official mechanisms for reporting violence, how do we learn about this CRSV at the time? How can we establish the forms of protection that may facilitate prevention if we are always analyzing well after the event? Scholars may be able to use and adapt a range of approaches, including new data science methods, to document where CRSV is occurring in immediate time frames, and, when and where CRSV is being reported or not. We suggest that news media reports are an untapped data resource to document CRSV. While local media coverage may replicate the exclusions and biases of official CRSV reports, the daily volume and accessibility of news reports provide an opportunity to redress some existing bias and exclusion as well as time lag. In some contexts, local media reports may be limited and/or published via international outlets because of danger and harm in contexts with lower press freedom, internet penetration and repression. A global news source, such as Factiva consolidates local, regional, and international reports in multiple, if not all, languages.

We acknowledge that prevalent gender norms and cultures of stigma regarding CRSV are pervasive and affect all forms of reporting, including by local media. For instance, news outlets make choices about which stories they cover, what angle they emphasize and which sources they speak with; they may “reinforce dominant stereotypes that are rooted in patriarchal conceptualisation of gender roles and sexualities” (Baker & Manchanda Rodrigues, 2024, p. 200; De Benedictis et al., 2019).

Despite the limitations of media reports and media framing, “there is no absolute, independent ground truth on which events have occurred and should have been covered,” and thus, possible media selection bias “should be understood as relative topic coverage, which is a narrower notion” (Huang et al., 2024, p. 10). Media reports are a significant data source currently missing from CRSV understanding and analysis. Their sheer volume means that we can collect thousands of reports on CRSV events “to parse variables worth exploring in the data” using our conceptual and context-specific knowledge (Goldberg, 2015, p. 2). The ML and GenAI analysis of media reports will not overcome all bias, but they give us powerful additional tools to include this data and find patterns in events to further examine. Official reports have intrinsic normative, legal, and political value; however, the systematic analysis of news reports on CRSV has been neglected, as has ML technology available to track and prevent CRSV in real-time. Above all, like McPherson et al. (2020), we argue that plural approaches to human rights fact-finding are needed. Moreover, given the difficulties with documenting CRSV in particular, novel data science methods for the collection, analysis, and triangulation of reports should be considered.

Piloting GenAI and ML Methods for CRSV

News articles and open-access humanitarian reports (see sources below) were collected for eight countries with different types of conflicts. Our sample of eight countries was selected because they were among the countries that recorded the highest incidence of conflict deaths due to state-based violence, non-state-based violence, and one-sided violence as recorded by the Uppsala Conflict Data Program (UCDP, n.d.) between 2010 and 2020. From an original list of 20 countries, we narrowed the selection to two countries per region, ensuring variation in the three types of political violence (state-based, non-state based, and one-sided). Because terrorism is a type of political violence but not explicitly coded by the UCDP we cross-checked the UCDP top 20 list with the Global Terrorism Database variable for total number of terrorist attacks by year to ensure our country selection was inclusive of terrorism as a type of violence. Our final selection of eight countries were Myanmar and the Philippines (Asia), Syria and Iraq (Middle East), Nigeria, Central African Republic and South Sudan (Africa); and Colombia (Latin America). Due to the higher representation of conflicts in Africa, and one conflict country for Latin America, we included one additional country from the African region.

For the feasibility of coding, between 2000 and 2020, we selected a period of three years for report collection determined by the highest phase of conflict intensity: one year before the peak of violence, the year with the highest incidence of violence, and one year later to study the phases of conflict. To control for the presumed relationship between political violence and CRSV, we selected three out of the eight countries that were not recorded in the annual UN CRSV list of affected countries.

Manual Coding Process

Our initial manual sample consisted of 11,900 articles reporting “sex crimes” sourced from Factiva Analytics (News aggregate source) and ReliefWeb (media, international organizations and non-government organizations reporting site) (see Appendix A). For each report of CRSV, we manually extracted information to identify where the event occurred, the type of CRSV, the characteristics of the victim and/or survivor, and importantly the number of SGBV crimes, when and in which location the event occurred. Assuming a 15-min analysis time per article, we estimated that it would take approximately 20 months for a full-time manual coder and 20 months for a part-time coder to analyze Factiva and ReliefWeb reports, respectively. In addition, manual coding required numerous sessions among coders to learn to correctly apply the codebook and further Q&A sessions to refine the codebook rules. This process partly, though not completely, addresses issues of intercoder reliability. Maintaining consistency in adherence to the codebook among numerous human coders is challenging. Highly complex reports are susceptible to subjective interpretations during the coding process. For example, in the Philippines reports, we found several events about police abuse of LGBTI people. However, in these cases, it was difficult to define an explicit link between those crimes and the conflict context. The potential of artificial intelligence (AI), particularly ML methods, to automate the coding process seemed a promising alternative. These methods offer solutions to the inter-coder reliability issues inherent in manual coding and enable the expansion of coding analysis to the desired sample of countries.

The automatization process aims to extract text and critical information from current news articles about key CRSV variables to understand their patterns. Factiva's reports were deemed to be more up-to-date or had less time lag between the time the events occurred and the time they were reported in Factiva compared to ReliefWeb. Factiva is one of the largest current international news databases (Dow Jones and Reuters) that aggregates news reports from major and regional news articles, industry news, and newswires from 200 countries in 30 languages.

We decided to test the automatization process on an expanded set of Factiva reports, more voluminous than possible to manually code. We aimed to extract and analyze text about CRSV events from Factiva news articles through technological tools that optimize the processing time of large volumes of information and standardize the extraction process for the variables. Twenty-six thousand news articles were retrieved from Factiva, for the eight country cases over 20 years, from January 1, 2000 to December 31, 2021. Each article's automation was implemented using AI, which allows word extraction through learning and linguistic theory methods. The extractions were represented in knowledge graphs that allow us to identify different variables of the events. These variations were processed and encoded in spreadsheets, and we determined merging rules to identify single or multiple events in each article.

For the knowledge graph construction from news reports, we employed two models ML-based model and LLM-based GenAI model. This process involved four phases: First, the preparation of the parameters for the automated process by converting the codebook into an ontology model. Second, the automation of the manual coding process using ML-based and large language model (LLM)-based GenAI models. Third, the testing and validation process involving re-calibration of the AI algorithm. Fourth, the production and evaluation phase of the coded outputs. In the next part of the article, we describe these methodological steps in automating coding of CRSV media reports.

ML for Understanding CRSV

Phase 1: Data Preparation

The codebook for the manual coding of CSRV events in Factiva reports was refined so that it could be applied to the ML pilot. We removed some variables from the database (URL of original reports and coding for “other human rights violations’’) and added two new variables (name (s) of the perpetrator (s), i.e., ISIS, and the date or date range when the events happened, i.e., since a month) to support the validity of the extracted information. Examples of different perpetrators and types of sexual and gender-based violence were also added from the original codebook to help clarify and refine the codes (see Appendix B, supplementary material).

We read and analyzed a total of 284 articles, of which 144 (51%) were manually coded as containing CRSV. The next step was to compare the manual coding with the results delivered by the ML tool. In the test sample, it was found that for 96 articles (67%), ML had also extracted information on CRSV events or victims. Both manual and automated extraction were pulled together for each article, and we found 148 matching events.

Analyzing the database with CRSV events at scale with ML requires a technological tool capable of supporting analysis and text extraction from the Factiva articles. We analyzed different techniques of natural language processing (NLP) and AI to obtain knowledge graphs and retrieve relevant information from the Factiva articles. We identified the text analytics platform, the extraction tool which consists of NLP pipelines that process unstructured data (raw text) to extract and assign structure and use NLP models to assign meaning to parts of sentences (Li et al., 2024; Zhuang et al., 2023). The ML tool extracts candidate word tokens and phrases using deep learning and linguistic theory methods to provide high probability predicate matches to a target ontology. This approach leverages advancements in NLP and deep learning to interpret and structure unstructured text (Asudani et al., 2023).

The information processing using Text Distil software was carried out for English articles in the eight countries in this study. Based on the codebook, we analyzed only sexual and gender-based crimes related to conflict, excluding general statements about sexual and gender-based crimes that were unrelated to a conflict event. Thus, we excluded reports about intimate partner violence, sexual violence related to the church or politicians and online sexual harassment. The pilot dataset was randomly split into a training set (80%) and a testing set (20%) to tune and test the NLP engine. Output accuracy of 80% or over was our target goal of the pilot study.

To enable the extraction of CRSV knowledge from the Factiva texts using the extraction tool, we built an ontology. In AI, an ontology refers to a shared vocabulary for researchers (Goled, 2021). It includes machine-interpretable definitions of basic concepts and the relationships between them. Ontology-based AI allows the system to use content and relationships among the content to make inferences that emulate human judgment. We also defined a data taxonomy along with the ontology, which classifies data and sub-categories. ² For example, in an event, a person could be a victim or a perpetrator with a gender, age, religion and ethnicity, among other attributes. For each attribute associated with a person, we define if male, female or non-binary, adult or child, and different possible religions and ethnicities. Both ontology and taxonomy were detailed for each set of variables and gradually perfected as the team became more familiar with the project.

We also implemented the event identification flow (EIF), a partially ordered graph whose objective was to support the identification of SGBV events in the text, define the extracting of associated information with each event, such as perpetrator, and finally, provide the merging rules for the same events into one. Overall, this preparation phase was one of the most multidisciplinary and challenging in this pilot study.

Phase 2: Automating the Knowledge Extraction

To facilitate ML, knowledge extractions of relevant text about CRSV events were inserted into the knowledge graph, which is an RDF triple store, which is then made available for downstream processes. Next, a series of queries are applied against the knowledge graph to extract the knowledge facts about the CRSV variables and their attributes from the triple store. Finalized variables are then used to code the “reporting rows and columns” of the final output based on the codebook.

As social scientists we shared the inputs and analysis for the project with the data scientists on the team: the Codebook, ontology, taxonomy, EIF, Pseudocode, spreadsheet template for the automated outputs, and comparison of the manual coding of the Philippines (26 articles and 40 events) and Myanmar (431 articles and 734 events) news reports for training purposes to support the detection of events in the text. A first run of the extraction tool using information from a subsample of this training data was performed, configuration changes were made and verified weekly with a set of eight to 10 articles to validate the improvements in the quality of the text extraction. Conceptual issues related to CRSV had to be clearly explained at this stage to the data scientists, for example, the difference between sexual violence and gender-based violence, the reporting distinction of a girl from a woman. After the first ML runs for the Philippines and Myanmar, and the testing using hypothetical news reports, we refined the extraction tool and improved the text extractions’ performance. Using the EIF and the pseudocode as a guide, the implementation of the event merging logic was defined. In this implementation, only consecutive events (i.e., one event followed by the next immediate event) were merged. Crucially, the extraction tool adopted a semantic similarity search approach rather than a list of pre-defined keywords for each desired variable. We generated separate lists of synonyms for the variables as well as lists of attributes by country for variables such as religion, ethnic groups, state armed, and non-state armed groups.

Preparation for the final run of the extraction tool included removing duplication and bugs from the Factiva news reports for the eight countries. To manage duplicate reports, Factiva deduplication settings enable users to filter out all duplicate articles, virtually identical articles or similar in content articles. We also performed process checks to ascertain the similarities of articles. An article was removed if the similarity score was greater than the pre-defined threshold. In total, 36,636 articles were extracted from Factiva. After the removal of duplicates and bugs, 75% of the news reports were included in the final ML run, as shown in Table 1.

Table 1.

Post-Preparation Factiva Articles for Machine Learning.

Country	Total Articles
Central African Republic	864
South Sudan	1,429
Philippines	5,580
Myanmar	2,033
Colombia	1,052
Nigeria	12,526
Syria	1,113
Iraq	2,739
The total sample for the pilot	27,336

For the LLM-based GenAI model, this lengthy phase was bypassed because GenAI does not need a training dataset. As we explain next, phase three of the study involved providing just a few examples in the input prompt, utilizing “In-Context Learning,” to extract the necessary information about CRSV variables.

Phase 3: LLM-Based GenAI Models for CRSV

The performance of automated ML methods in extracting key information about CRSV from media reports is promising. But it still requires a complex procedure and NLP extraction tools. Recently, notable progress has been achieved within the realm of GenAI using large language models (LLMs). Specifically, the latest LLMs, such as OpenAI's ChatGPT and GPT-4o (Achiam et al., 2023), have demonstrated remarkable training-free instruction following and reasoning capabilities that are comparable to, and in some cases, even surpass human capabilities.

Despite numerous remarkable breakthroughs, the hallucination problem associated with LLMs raises significant concerns. Hallucination refers to generating content that strays from factual reality or includes fabricated information. To enhance the relevance and accuracy of the outcomes, we used methods to mitigate the hallucination problem inspired by prompting approaches such as in-context learning (Brown et al., 2020), Chain-of-Thought (Wei et al., 2022), and Retrieval Augmented Generation (Lewis et al., 2020).

By prompting LLMs, we can improve their capabilities to accurately extract and structure information from the Factiva media reports. We conducted three types of prompting for information extraction from reports: In-context learning (ICL), which leverages the concept of “learning by demonstration,” helping the LLM to recognize and replicate tasks by giving it examples on the precise type of information to extract, as shown in Figure 2; chain-of-thought (CoT) prompts which provide step by step reasoning to guide the model's accurate extraction of complex information before responding as shown in Figure 1; and retrieval-augmented generation (RAG), which combines the LLM with an external retrieval system to integrate relevant documents or data, enhancing the extraction. For each conflict-related media report, we extracted information to identify where the event occurred, the type of violence, the age and gender of the victim (s), and the estimated number of sexual crimes. In our experiments, we conducted a comparison between the automated ML method, GPT-4o with few-shot prompting and with CoT prompting methods for six variables (type of sexual and gender-based violence (SGBV), age of victim, timing, gender of victim, SGBV count, and site of violence).

Figure 2.

Performance of GPT-4, GPT + CoT and Automated LM Models for CRSV Information Extraction.

Note. CoT = chain-of-thought; CRSV = conflict-related sexual violence; LM = language mode; SGBV = sexual and gender-based violence.

Figure 1.

GenAI Prompting.

For the six variables coded from CRSV media reports, we compared the performance of LLM-based GenAI from ChatGPT to the most recent iterations of GPT-4 and GPT-4o with CoT prompting of the automated ML model. Figure 2 shows the accuracy results of the different approaches for each of the variables.

Phase 4: Validating GenAI and ML Models at Scale

For the ML model, the news reports were divided into six groups according to their word length to optimize the processing time of the information on the servers. As a result, the extraction tool processed approximately 26,300 Factiva articles (96%) under 2,000 words for the eight countries. It generated knowledge graphs and event record spreadsheets for each. For the GenAI model, there was no restriction on the length of the input document because we had configured the LLM to handle up to 4,000 tokens (approximately 3,000 words).

To assess the quality of the outputs delivered by ML, we randomized analysis of 2% of all articles from six out of eight countries: Colombia, Syria, the Central African Republic, South Sudan, Iraq, and Nigeria. ³ We analyzed at least one article in each set with an extended word length that represented a higher level of complexity in terms of analysis.

As shown in Table 2, we found 148 matching CRSV events, and on average, 87% of the CRSV variables and attributes identified through manual coding were extracted by the ML model. CAR and Nigeria are the countries with events with the highest percentage of correct extractions, while Iraq and Colombia had events with the lowest percentage of correct extractions.

Table 2.

Percentage of Correct Extraction Overall.

Country	Total Comparable Events	Total Events, Including Split Events	Percentage Correct Extractions Overall
			Lowest (%)	Higher (%)	Average (%)
Colombia	24	24	79	93	88
Syria	6	6	83	91	87
Central African Republic	17	18	81	96	89
South Sudan	26	29	80	95	87
Iraq	47	50	77	92	85
Nigeria	28	28	83	96	89
Total	148	155

We manually reviewed whether the information automatically extracted for each of the 71 attributes was accurate or not (i.e., if information about the sex of the victim in the text was coded the same as by a human). For each variable (i.e., 2a. location), we identified the number that were correct or incorrect extracted by the tool (i.e., urban, rural, or unknown). As shown in Figure 3, five variables (type of SGBV, gender of victim, gender of perpetrator, SGBV count, and location of violence) could be automatically coded with accuracy greater than 70% and a further six variables with accuracy between 50% and 70%.

Figure 3.

Accuracy of ML Extractions of CRSV Variables.

Note. ML = machine learning; CRSV = conflict-related sexual violence; DV = domestic violence; ID = identity of perpetrator; SGBV = sexual and gender-based violence.

The most robust ML extraction of CRSV variables was the type of violence and the gender of the victim. For the feasibility of the CRSV analysis, prioritizing those variables most amenable to large language models is crucial. Because the selection of variables may affect the understanding of CRSV patterns, however, it is especially important to prioritize those variables that, from a CRSV conceptual perspective, are vital to understand (e.g., victims, type of violence) or are typically not recorded in official reports (e.g., where CRSV occurs). We identified specific limitations with the validity of the ML coding of some of the more complex variables and attributes of CRSV despite efforts to adjust the algorithms to improve their accuracy. We discuss these in turn.

SGBV count: Much CRSV scholarship has focused on counting acts of sexual and gender-based violence (SGBV), defining widespread and systematic sexual violence in terms of the scale of the violence and those affected. Even for manual coding of reports, however, counting SGBV is extremely challenging. For our trained ML model, this variable achieved over 70% accuracy, but there were limits to its accuracy. Many articles reporting SGBV in conflict settings make an overall count of victims, that is, they describe the aggregate number of “women and girls” who were rape victims. The automated extraction tool, however, separated the information for women and girls. Manual oversight was required to ensure that one of these entries was not excluded from the overall SGBV count or that the victims were counted twice. Other challenges involved the extraction tool identifying the number of perpetrators or the victim's age as the SGBV count.

SGBV victim age: We aimed to identify if the victim was an adult, teenager, or child with automated coding. However, we found that the differentiation between children and teenagers generates coding noise since it is common for media reports to refer to the victim as, for example, a “14-year-old girl.” To improve the outcomes of this variable, we collapsed it into two categories: adults (18+) and children (18-), with teenagers included as children.

Automating coding of the timing of SGBV crimes requires more consideration of how to identify in the text of the news report the precise date or range of years when the crimes happened. Months, years, or expressions related to time (such as last week, last year, or since March) must be extracted from the text to approximate this variable with greater precision. This involves distinguishing the date of publication of the report from the date on which the crimes reportedly happened. Even with real-time news, there is often a time lag between the event and the reporting of it.

Further coding irregularities arose for site, region, and timing, which are more difficult to resolve. When different locations of violence, such as a road, school, or camp, were named in a news report, the ML tool would associate all of them with the site of the SGBV event, which, upon manual review, was not the case. Similar results occurred for region, where the extraction tool identified locations such as camps, huts, trees, roadside, and bedrooms as the region. By providing GPT-4o with examples in COT prompting we were able to train the AI tool to identify the location of the SGBV crimes with above 70% accuracy.

Finally, the five variables below 50% ML accuracy included those related to the perpetrator, including whether the perpetrator is an armed combatant or civilian (ID) and identifying perpetrator affiliation with a pre-selected list of combatant groups, which are important for insight into the relationship between CRSV and political violence. False positives arose due to false associations of perpetrator ID and affiliation. The more complexity within a report, the greater the likelihood of these errors. Two attributes under perpetrator affiliation failed to be satisfactorily identified: Terrorist affiliation and international affiliation, the latter of which is relevant especially for state armed groups in foreign countries, such as British soldiers in Iraq. In 55% of the events, the ML extraction tool performed a successful extraction of the name of the perpetrator but did not correctly identify their affiliation as state, non-armed state group or unknown.

To analyze current patterns of CRSV and their relationship to political violence, we recommend prioritizing five variables: type of SGBV, gender of victim, gender of perpetrator, SGBV count, and location of violence. Qualitative case study research is needed, however, to analyze the broader context of human rights violations in which CRSV occurs. Understanding the continuum of violence as well as causal relationships is too complex to discern from ML information extraction via media reports. CRSV analysis using GenAI can be replicated by other researchers, as with manual coding, via a GPT-4o codebook that includes the COT prompts, which are informed by CRSV and context experts, as is the case with our CRSV GPT-4o Codebook (True & Davis, 2025). Findings can be generalized to other situations and countries by using similar CoT, though using different contexts and language-specific terms.

Methodological Advances and Limitations

Our pilot study of ML and GenAI data analysis of news reports about conflict-related SGBV allowed us to test a new method. Manual coding is time-consuming and fallible, considering the volume and length of news articles reporting violence, as discussed in the introduction to this article. The sheer amount of time and labor required for humans to code news reports may limit the timely analysis and application of insights from the data into active conflict situations to inform CRSV prevention. There are also issues with the accuracy and the selectivity of formal reports that compile received submissions. The study results showed that ML model and especially GenAI GPT-4o with prompts are powerful tools for the extraction and analysis of qualitative information for three main reasons.

First, they allow for significant volumes of reports to be rapidly coded, overcoming the significant time required for manual coding to determine the presence or absence of CRSV. Second, with training for ML and/or CoT prompting for GenAI informed by CRSV human experts, they can produce more reliable coding at scale. For example, ML and GenAI achieved consistency in the extraction and coding of simple variables such as the gender of a victim for thousands of reports. With this method, researchers can overcome the significant issues of inter-coder reliability and generate large datasets by using ML and GenAI tools for current patterns of CRSV focused on five variables: type of SGBV, gender of victim, gender of perpetrator, SGBV count, and location of violence.

Third, because of the greater reliability, GenAI may produce less biased, including less gender-biased, data than humans. To generate an understanding of the patterns in a conflict-affected context of who perpetrates what type of sexual violence, and/or is a victim of it, and where it takes place, you need to code a lot of reports, which inevitably leads to human error or inconsistency across human coders. The extractive tools of ML and GenAI can be trained to replicate the same analytical process of extraction for all variables and attributes of CRSV events in a report.

Scholars have investigated the possible gender bias of LLM-based AI. AI can import stereotypical beliefs about gender from training data, which can further import biased reasoning related to gender, not only in information retrieval but in decision-making (Levy et al., 2024). This could potentially impact, for instance, the kinds of victims and perpetrators identified in data extractions. However, as we have explained, prompting LLMs can improve their capacity, including their gender inclusivity. This underscores the important role of CRSV subject matter and context experts collaborating on the GenAI codebook, including the COT prompts.

While the reliability of ML and GenAI analytical coding is impressive, the validity of automated coding is still limited. For simple variables of CRSV reports, they performed well, but for more complex variables, their accuracy cannot match manual coding. Some challenges are shared. In a single news report, it is often difficult for a human or machine to identify single or multiple sexual violence events. Moreover, there are still limitations that need to be addressed to improve ML and GenAI results. For example, many news reports in our sample provided either the perpetrator or the victim's name from which their sex/gender can be inferred by a human, manual coder. If “Mary” is identified as a female name for the coder, then her gender is a woman. This was not possible with the ML extraction tool we used and produced many errors given the diversity of languages and names. For the machine to make that inference and to extract the variable correctly, it requires an extensive list of names (by country and language) with their respective gender. Operationalizing the codebook for ML or GenAI is a distinct step in the method, which will involve developing a taxonomy of important terms to use in CoT for GenAI (or the Ontology model for ML).

ML and GenAI data extraction can be fast, but a further challenge is that basic machine errors can also lead to both false positives and false negatives. Manual review of automated coding is necessary to investigate these errors. It can address the potential for data miscalculations on the scale and nature of the problem, as the perpetrator errors revealed in our study. Even with the best efforts of forensic social and data scientists and the most up-to-date technology, variables that require inference from the surrounding context and “text,” such as motivation for the sexual crime or the number of victims, will be less accurate. These variables require qualitative analysis based on manual coding—and therefore a mixed-method approach.

In sum, manual coding is still crucial in some elements of the ML and GenAI process. Human expertise in sexual and gender-based violence and the conflict contexts in which it occurs is essential to assist with ML event detection, ontology, and merge logic as well as prompting of GenAI applications. The value added is the chance to collect data in concurrent time frames to conflict. This approach allows scholars analyzing CRSV to complement large-N-data with specialized reports from reputable organizations and/or contextual information with immediate analysis of the types of CRSV and the factors driving it.

As well as being advantageous for scholars, data science methods for analyzing CRSV have policy and decision-making implications beyond academic research. Scholars agree that big data ML and AI methods can assist policymaking especially in data-driven governance areas such as crisis-management, environmental regulation, education, and health (Giest, 2017). Currently, the causal inference capacity of GenAI limits its use as regulatory evidence (Kim, 2024). Currently, the Office of the United Nations High Commissioner for Refugees's (UNHCR, n.d.) Project Jetson uses predictive analytics to forecast forced displacement of people. Since 2021, the United Nations Development Program established a complex risk analytics fund to support action-focused data collaboration and data-derived insights that could be translated into crisis action to improve rapid responses and resource mobilization in global emergencies. The Violence and Impact Early Warning System—People in Need project already uses ML and Gen AI learning to forecast humanitarian need for all low- and middle-income countries (UNDP, n.d.). However, the accuracy of such tools requires ongoing attention to ethical, safe, and secure reporting practices that gather information from and about victims-survivors (Beduschi, 2022; Institute for International Criminal Investigations, 2022).

The use of ML and AI data science methods in the CRSV field could have a significant impact on prevention as well as responses to this violence. Earlier information on CRSV patterns could be secured to support decisions about the deployment of humanitarian and peace operations responses to CRSV. ML and AI are already being used to predict and forecast displacement patterns, to improve humanitarian service delivery, and target populations for protection. With AI tools and access to media reporting data, the recognition of CRSV “situations of concern” by states, could potentially be expedited to the UN Security Council, the International Criminal Court, the International Court of Justice, and the UN Human Rights Council. Monitoring to inform ongoing human rights investigation and reporting has been underway for some time with the International, Impartial and Independent Mechanism to Assist in the Investigation and Prosecution of Crimes Investigation and Prosecution of Persons Responsible for the Most Serious Crimes under International Law Committed in the Syrian Arab Republic (IIIMM) (Abdulrahim, 2021). ML and GenAI analysis could bolster criminal investigations by identifying broader patterns of CRSV occurring at the same time and location as the crime(s) under investigation.

Like all reporting of sexual and gender-based violence in conflict situations, there are potential risks of harm to victim-survivors and populations, and of greater insecurity and stigma due to the reporting. It may be difficult to achieve inclusive humanitarian protection with any reporting. Data science methods do not solve the problem of CRSV per se. However, the lack of formal reports on SGBV in a variety of conflict contexts, and at its earliest onset, maintains silence and impunity for this violence. Research using data science can shine a light on additional situations and new CRSV patterns to justify responses, if not of the granularity required for legal prosecution, but for more immediate prevention and protection from this type of violence.

Conclusion

This article began with a problem: Understanding the patterns of conflict-related sexual and gender-based violence (CRSV) is limited by both the absence of reporting data and the bias or exclusion in existing datasets. Data collection is also beset with the real problem of what to include and exclude. But these choices may limit our knowledge of situations of CRSV in unintended ways. Repertoires of sexual and gender-based violence in conflict settings are complex, large, and unwieldy. We studied media reports from eight different conflicts over a decade to see if they could provide an alternative, more immediate source of data than official reports addressing some of the gaps in our knowledge. Media reports are a novel, global source of information about CRSV. Their analysis has the potential to be one more tool to address the absence of CRSV reporting data.

We compared the utility of ML and GenAI analytical techniques for extracting CRSV events and variables to see if they could overcome some of the challenges with manual coding of CRSV reports. Our findings show that there is promise in automated data science approaches applied to of unofficial media reports to improve the analysis of CRSV. GenAI is most promising with the use of prompts in GPT-4o, the most recent and advanced publicly accessible GenAI model, greatly enhancing the speech and accuracy of analysis. The experimental process described in this paper revealed that LLM-based GenAI can rapidly and accurately identify and code the type of sexual and gender-based violence, the gender of victim and the perpetrator, the count of sexual and gender-based crimes, and the location (site) of violence.

ML and GenAI requires access to media reports, and their large-scale text-based data was provided by an expensive user subscription to Factiva Analytics for research purposes. This is a factor to consider in evaluating the cost-effectiveness of manual versus automatic coding, and the benefit of analyzing media reports on CRSV at very large scale: In this pilot study, almost 30,000 reports. We found that a multidisciplinary expert team with both data science and social science knowledge is vital to conduct and validate automated coding via ML, which can also increase the cost of this research. With the advent of GenAI, however, the time-intensive training of a tool is not needed. As such, the costs of research are reduced, and the ease of automation is greatly enhanced. Indeed, social scientists can readily master the prompting approach with GPT-4o and later versions to improve the validity and relevance of the CRSV analysis of media reports.

The main lessons learned from our study are two-fold: Social and political scientists need to have modest expectations about the number and complexity of variables that can be analyzed with ML and GenAI. In addition, we should recognize that for sensitive, inherently complex topics like CRSV, human-centered investigation and contextual analysis will always be an essential complement to data science efforts to understand the unique social and environmental dynamics of violence as well as its broader patterns.

Author Biographies

Sara E. Davies is professor of International Relations at Griffith University, Australia and deputy director of the Australian Research Council Centre of Excellence for the Elimination of Violence against Women (CEVAW). Her research focuses on how to scale up prevention and protection capacity for at-risk populations during crises, including conflict, mass atrocities, and infectious disease outbreak events. She is currently serving as a member of the WHO Technical Advisory Group on Health, Migration and Displacement (2025–2027). Recently, Professor Davies has co-authored Hidden Wars: Gendered Violence in Asia's Civil Conflicts (Oxford University Press, 2024) with Jacqui True, and The Dynamics of Conflict-Related Sexual and Gender-Based Violence: Comparing Impact Across Global Regions (Davies et al., De Gruyter, 2024).

Jacqui True is professor of International Relations at Monash University and director of the Australian Research Council (ARC) Centre of Excellence for the Elimination of Violence against Women (CEVAW). She is also a global fellow of the Peace Research Institute, Oslo (PRIO). Her research focuses on gendered political violence, feminist political economy, and the Women, Peace and Security agenda. Professor True is the author of 22 books and over 150 articles. Most recently, she co-authored, Hidden Wars: Gendered Violence in Asia's Civil Conflicts (Oxford, 2024) with Sara E. Davies and The Dynamics of Conflict-Related Sexual and Gender-Based Violence: Comparing Impact Across Global Regions (Davies et al., De Gruyter, 2024).

Fatemeh Shiri is a lecturer in the Department of Data Science and Artificial Intelligence at the School of Computing Technologies, RMIT University. Prior to joining RMIT, she spent four years as a research fellow at Monash University. Fatemeh earned her Ph.D. in computer science from the Australian National University in 2021.

Yolanda Riveros-Morales was previously a research fellow at Monash University. She is currently data methods and design manager at the International Women's Development Agency in Melbourne, Australia. Yolanda received her Master of Economics from the Universidad Los Andes, Colombia in 2012.