Data-usage descriptors as search metadata: the case of food security data and the National Data Platform

Abstract

Scientific data is a critical input into scientific research. Yet the research data landscape is constantly changing as new datasets emerge, others are retired, or some disappear altogether. Without a systematic way to track how datasets are used across a research field, researchers have no reliable method for identifying relevant data resources or locating communities that work with them. Data-usage descriptors can substantially advance research productivity by reducing the time that researchers spend finding new and relevant datasets in their research field, and the communities that use them. This paper describes how to generate data-usage descriptors by finding how datasets are used in publications and then linking the dataset information to the publication metadata. It also shows how usage descriptors can be used to find other related datasets and their usage. It concludes by arguing that the approach represents a critical piece of foundational infrastructure that could be deployed in repositories as part of a referenceable, navigable, and contextual data framework. This article contains a reproducible workflow for constructing data-usage descriptors, based on analyzing the full text of publications in the Dimensions database. The illustrative use case is research on food security. The illustrative repository is the National Data Platform.

Introduction

Scientific data is a critical input into scientific research. The research data landscape is constantly changing as new datasets emerge, others are retired, or some disappear altogether. New data can arise for many reasons, including deliberate investments in specific fields such as the Protein Data Bank¹, ImageNet², or the Sloan Digital Sky Survey³, or the emergence of superior data collection technologies⁴ or new scientific challenges, such as the novel coronavirus⁵. Datasets can also disappear as a result of deliberate disinvestment decisions, such as the 2023 cuts to the US National Ecological Observatory Network⁶, or lack of use⁷, while others are inconsistently collected due to funding and implementation challenges as in the case of the Survey of Inmates in Local Jails⁸. In recent months, US federal datasets have been quite rapidly retired⁹, affecting the productivity of both scientists and policy researchers in a number of fields, including food security.

The constant churn in scientific datasets creates a fundamental challenge for empirical science. Datasets, even if placed in well curated scientific repositories, are typically not cited using standard methods – only about 1% of datasets indexed in DataCite have at least one citation¹⁰. Thus, researchers in research fields characterized by such churn neither have a systematic way of discovering new or alternative data sources in their fields, nor of finding the relevant user communities. As a result, the scientific impact of the disappearance of datasets is difficult to quantify and the potential value created by emerging datasets may not be fully realized.

This paper proposes an approach to address that challenge by developing data-usage descriptors based on finding instances in which researchers use data in scientific publications. It applies natural language processing techniques to extract information about dataset use in the full text of publications. It then makes use of the existing structured metadata about publications – authors, their institutional affiliations, field of research, citations, and journals – to intentionally develop descriptive metadata about data use. The paper proposes that the resulting “data-usage descriptors” could be used as an additional source of information about the research data system and enhance the empirical study of science and innovation.

There are many potential scientific opportunities that could be realized if such data-usage descriptors were included in the data knowledge infrastructure. Borgman and Groth, for example, identify four distinct groups: data creators, data reusers, data archivists and funding agencies¹¹. Data repositories or other data infrastructures could use them to complement familiar indicators of scientific activity, such as publication and citation counts or grant productivity by characterizing how the datasets in their repositories are used^12,13. They could provide evidence to the funders of data about the scientific value of investing in data by quantifying which parts of the data infrastructure are heavily relied upon, which communities depend on them, or which parts of the data ecosystem are vulnerable when datasets are discontinued. Another set is social. Quantifying dataset use could create an incentive structure to reward data producers for their contributions in a manner similar to Project CRediT^14,15. Intentionally connecting expert users could foster reproducibility and replicability and reduce the cost and errors of siloed data efforts.

The illustrative use case for this paper is producing data use descriptors for food security research data. The use case is an important one in its own right: empirical research on food security is used to inform decisions that affect almost 50 million Americans, including 14 million children, particularly those who rely on public benefit programs¹⁶. But it is also important because of the high profile elimination of one of the major data sets for food security researchers: the Household Food Security Survey Module, or HFSSM¹⁷. That dataset has served as the official measure of food insecurity for three decades^18,19. That proposed elimination alerted the food security research community to the importance of quantifying the consequences of dataset churn.

The paper goes beyond simply proposing a method for developing data-usage descriptors for a known list of datasets within a definable research community. It also outlines how the methods can be expanded to find alternative data sources and to generate data-usage descriptors for those sources. It provides code and a workflow that can be replicated and enhanced for any field with an identifiable core set of datasets. Indeed, the workflow described here has been implemented in two other domains – workforce data used in the labor economics domain, and pediatric asthma from the health sciences domain – for another project examining how data-usage descriptors can inform the development of data trust based on user assessments of data quality. That application is also briefly described in the paper.

The paper also provides a concrete description of how data-usage descriptors have been produced and been deployed in a new data repository funded by the National Science Foundation – the National Data Platform, or NDP^20,21. It describes the potential to add a “contextual data-insights service” to the NDP that will add data-usage descriptors to the NDP’s support of data user needs for data-driven scientific workflows, such as ease of access and use, time to science, and energy/environmental impact^20,22. It includes a discussion of how data-usage descriptors could be adopted at scale by other critical data infrastructures that are hosting new types of data.

The paper concludes with a discussion of opportunities and limitations, with a particular focus on directions for enhancing the search methods and incorporating human engagement.

Methods

There have been a number of attempts to link datasets to papers. One of the most well-known is the dataset knowledge graph based on identifying dataset mentions in abstracts and citation contexts in the Microsoft Academic Knowledge Graph²³. Two working groups in the Research Data Alliance (Scholarly Link Exchange and Data Usage Metrics) have been developing standards and frameworks to link articles and datasets in order to develop data metrics²⁴. A recent conference “The Make Data Count Summit” provides a rich overview of actionable next steps, including the work described in this paper.²⁵ Google Dataset Search crawls the web and has identified 45 million datasets from 13,000 sources²⁶.

This paper draws on both the previous work and on our own decade-long effort to develop data-usage descriptors. We started more than a decade ago with relatively primitive Natural Language Processing(NLP) techniques²⁷ to determine whether it was possible to find datasets in publications. We found funding to pay six annotators to work for almost six months to manually read and annotate a set of 10,000 open access publications and we designed a competition in 2018 which attracted 20 submissions in February 2019²⁸. Those techniques were improved by attracting more than 1600 teams across the world to develop improved search tools in a 2021 Kaggle competition²⁹. Both the tools and the corpus were then expanded and applied in the Democratizing Data project³⁰. The initial work on scientific research was expanded to include a focus on federal statistical data as part of a response to the 2018 Foundations of Evidence-based Policymaking Act³¹ required agencies to publish on their websites, on a regular basis (but not less than annually), information on the usage of public data assets by non-government users.

A number of methodological lessons were learned from this previous work and we draw on them in this paper. One lesson was that it is not only feasible, but extraordinarily useful, to use NLP to find datasets in publications and then use key publication metadata to describe dataset use. Those publication metadata, such as title, publication DOI, journal name, publication year, citation counts, subject classifications, authors, and institutional affiliations, can be used to produce a variety of data-usage descriptors. A contribution of this paper is to demonstrate that it is possible to produce data-usage statistics at scale across multiple domains. Technological advances in large language models (LLMs) have made it much more cost-effective to find datasets in large scientific corpora.

Another lesson was that the methods need to be developed in conjunction with research communities for a number of reasons. One is that the research community is needed to identify the initial seed datasets. Not only do r esearchers describe their datasets in different ways, but fewer than 20% of authors cite the datasets they use in a standardized format in the reference. Another is that research communities are also interested in distinguishing dataset mentions from dataset use. Thus some expert knowledge is needed to assess and validate initial results. A second contribution of this paper is an assessment approach that could be used by research communities to inform the further development of LLMs designed for dataset search and discovery.

Researcher named datasets

A fundamental challenge in developing data-usage descriptors for a dataset is that data users do not always use common terms to refer to the dataset, particularly as the distances between the data creators and data users increase^11,32. Researchers writing in scientific publications may refer to datasets in multiple ways, well before they are formally placed in scientific data repositories^10,33. In the case of new and emerging datasets, researcher practice is often to subset, update, version, fork and combine datasets, and to describe how the analytical dataset was constructed. The result is that there can be many ways in which a dataset can be described, and sometimes what a researcher considers a dataset does not fit in with sensible technical definitions. In practical terms, data users might not be as precise as data creators, and not differentiate between a dataset as “a collection of data items reflecting the results of such activities as measuring, reporting, collecting, analyzing, or observing” and a dataset page “a Web page that refers to a dataset” p. 340³⁴.

The focus of this paper is to describe an approach to constructing data-usage descriptors that could be added to existing metadata for given datasets. It is not to develop a global definition of a dataset nor to find all possible datasets that have been used for food security research. Thus we follow Renear et al.’s suggestion that a reasonable first step in defining a dataset is to rely on the “informal and general understanding of a disciplinary community” p.1³⁵. The team operationalized Renear et al.’s pragmatic approach by working with the food security community to identify a set of key datasets for which usage statistics would be of interest, and develop usage statistics for those. That list was identified, in keeping with the spirit of Renear et al.’s suggestion, through engagement with both USDA and the food security research community³⁶. Eleven researcher-named datasets were identified: Census of Agriculture, Current Population Survey Food Security Supplement (CPS FSS), Household Food Security Survey Module (HFSSM), Food Acquisition and Purchase Survey (FoodAPS), Agricultural Resource Management Survey (ARMS), Farm to School Census, Food Access Research Atlas (FARA), Circana (formerly IRI) Infoscan data, Local Food Marketing Practices Survey, Tenure, Ownership, and Transition of Agricultural Land (TOTAL) Survey, and Rural-Urban Continuum Codes (RUCC). More information about the datasets is provided in Supplementary Information.

There are several scientific corpora with full text^27,37,38. Digital Science’s Dimensions corpus³⁹ was chosen for the search because of its breadth of coverage, the quality of its curation, as well as the openness to scientific collaboration, and the shared scientific vision of the Digital Science team.

The Dimensions database consists of nearly 160 million publications with structured metadata about each publication, including inter alia the authors and their institutional affiliations, the research topics studied in the publication, and a running count of the citations, funders, institutions, and research topics.

Search process

The focus of the project was to document the workflow whereby data-usage descriptors could be developed from a given publication corpus for specific datasets. In order to make the workflow manageable, the team restricted the queries to publications published between 2015 and 2025 and authored by at least one researcher affiliated with a US institution, based on the Dimensions address mapping approach⁴⁰. These restrictions yielded a search corpus of 7,473,055 publications represented by distinct DOIs.

Digital Science provided access to the full text of publications through an application programming interface (API), which can be queried using Boolean operators in Dimensions’ Domain Specific Language. The query structure for querying Dimensions, as well as a complete list of input aliases for the food security datasets is fully described in Zenodo⁴¹.

A designated member of the team was provided with direct access to the Dimensions corpus via Google BigQuery, which allowed for automated and reproducible data retrieval. Where data analysis required access to full text beyond the scope of distribution licenses, this work was handled exclusively by Digital Science team members, and any data processing took place in secure environments entirely within Digital Science’s control.

The Dimensions partnership made it easier to develop and run the workflow reported in this paper, particularly because it provided access to full-text search across both open and licensed content. The underlying code, tooling, and workflow design are open including the conversion of pdfs to processable text using GROBID; exact reproduction of the reported results requires access to a source with comparable metadata and full-text search capabilities. The code and results were reviewed by all team members.

This section describes the approach used to search for the eleven “named” datasets in the more than 7 million publications. It also describes how “additional” food security datasets were identified. Each step is described and illustrated conceptually in Fig. 1 and discussed in this section.

Fig. 1.

Workflow to identify publications using named and additional food security datasets. Steps 1–3 describe how publications using the eleven named seed datasets were found. Steps 4–5 repeat the procedure to identify publications using additional datasets discovered through the workflow.

Finding named food security datasets

The team used a three-step approach to find publications mentioning the named datasets, sub-set the publications to those that studied food security, and then determine whether the authors used the datasets rather than mentioned them. Datasets and publications are the two units of analysis in this workflow, and each step is oriented around one or the other depending on the object of interest: some steps search and filter publications, while others evaluate the datasets referenced within them.

Finding publications (Step 1): The team used a string-search approach to identify publications in the Dimensions corpus that mentioned at least one of the eleven named datasets. Since researchers often reference datasets inconsistently, using acronyms, abbreviations, alternate spellings, or related URLs, we developed a structured list of dataset-alias pairs (“dyads”) to map each dataset’s official title to common variants. For example, the “Food Acquisition and Purchase Survey” was linked with the alias “FoodAPS.” Searches were executed in Python using the Dimensions API client dimcli⁴², and the results were returned as structured JSON objects.

Each full-text search took about five minutes. In the steps described below, the further LLM processing of the publication corpus of n = 2,321 on a Mac (M4 Max, 128 GB RAM) required roughly eight continuous days of runtime, which implies that applying the same deep analysis to the full Dimensions corpus of 7.4 million publications without prior filtering would be computationally infeasible.

Sub-setting to food security topics (Step 2): The team then filtered the publications identified from Step 1 to those relevant to food security using Dimensions’ concept relevance scores. In Dimensions, “concepts” describe the main topics of a publication, derived from the full text using machine learning⁴³, and each concept is assigned a relevance score between 0 and 1 indicating how closely the concept relates to the publication’s content. Each publication could be linked to multiple concepts. The two concepts of interest were “food security” and “food insecurity,” so publications with a concept relevance score greater than 0.6 for either concept were retained.

Assessing mention and use (Step 3): This step began with the set of food security publications from Step 2, each of which contains at least one string-match to a named dataset. A community of food security data users in a series of workshops noted that string search alone cannot separate substantive use from incidental mention, and such distinctions are important both for empirical work and for identifying data user communities⁴⁴.

Although previous work used manual validation approaches to distinguish mentions from use, modern LLMs are lower cost, more timely, more scalable, and provide rich contextual information⁴⁵. Accordingly, the team created a "dataset review agent” to review the full text of each publication to determine whether each matched dataset was substantively used in the research or simply mentioned.

The first prompt was an Assessment Prompt applied to the results of Steps 2, which assessed whether the mention was referring to the correct datasets. It then evaluated the named datasets against six criteria for substantive use. Those criteria were derived from the decade of experience with multiple corpora and community feedback^29,46,47, namely: 1. The paper performs analysis, modeling, or experiments on the dataset, 2. Results, findings, or insights are derived from the dataset, 3. The dataset is central to the research methodology, 4. Specific characteristics, variables, or samples from the dataset are discussed, 5. The dataset is used for data processing, statistical analysis, or machine learning, 6. The research methodology explicitly describes using this dataset.

The Assessment Prompt was applied to each publication-dataset pair identified by string search (Step 1). The LLM received the full publication text alongside the list of named datasets matched by string search and returned a ‘true’ or ‘false’ result for mention and use, together with an exact text quote and reasoning. The Assessment prompt was engineered as follows:

You are a specialized research analyst and expert at analyzing academic publications to identify genuine dataset usage. Your expertise lies in distinguishing between mere mentions and actual dataset utilization in research papers .

CONTEXT ABOUT DATASET IDENTIFICATION:

The datasets provided to you were initially identified through string search techniques that looked for:

- **Aliases**: Alternative names, abbreviations, or variations of the dataset name (e.g., “NASS Census” might also be called “Agricultural Census”, “USDA Census”, “AG Census”)

- **Flag Terms**: Related keywords, organizations, or terms associated with the dataset (e.g., “USDA”, “National Agricultural Statistics Service”, “Department of Agriculture”)

These string searches found potential matches in the publication text, but now as a specialized research analyst, your critical task is to determine if these datasets are GENUINELY UTILIZED in the research or merely mentioned .

VALIDATION CRITERIA:

A dataset is GENUINELY USED if:

1. The paper performs analysis, modeling, or experiments on the dataset

2. Results, findings, or insights are derived from the dataset

3. The dataset is central to the research methodology

4. Specific characteristics, variables, or samples from the dataset are discussed

5. The dataset is used for data processing, statistical analysis, or machine learning

6. The research methodology explicitly describes using this dataset

A dataset is ONLY MENTIONED (not genuinely used) if:

- It’s cited in related work or literature review without being used in the current study

- It’s mentioned as an example, comparison, or reference for future work

- It’s part of a background discussion without analysis

- It’s listed as a potential data source but not actually utilized - It’s mentioned in acknowledgments or funding sections only

YOUR ROLE AS RESEARCH ANALYST:

As a specialized research analyst, you must carefully examine the context of each dataset mention to determine if it represents actual usage in the research methodology, analysis, or results. Consider the publication’s structure (methods, results, discussion) and the specific role each dataset plays in the research .

DATASETS TO VALIDATE:

{dataset_list}

PAPER TEXT:

{text_content}

INSTRUCTIONS:

For each dataset, provide:

1. is_mentioned (true/false): whether the dataset’s exact name or one of its aliases appears explicitly in the text

2. is_used (true/false): whether the dataset is actively used/analyzed in the research methods, analysis, or results

3. The exact text quote that shows usage (if found)

4. Brief reasoning for your assessment as a research analyst

5. Context around the mention to support your evaluation

6. The dataset DOI (Digital Object Identifier) if explicitly mentioned in the text (e.g., “ 10.1234/dataset.123 ”, “ 10.1234/dataset.123 ”, or similar). Return null if no DOI is found. VERY IMPORTANT, ONLY RETURN DOI OF THE DATASET MENTIONED IN THE TEXT, NOT THE DATASET ITSELF .

STRICT RULES:

- Set is_used to true ONLY when there is DIRECT EVIDENCE in the text_quote or context that explicitly cites the dataset by its exact name or one of its aliases AND indicates it was actually used in methods/analysis/results .

- Set is_used to false if the dataset appears only via flag terms (e.g., organization names like “USDA”, “NASS”) or generic references without explicit dataset name/alias .

- Set is_mentioned to true ONLY when the dataset’s exact name or one of its aliases appears explicitly in the text_quote or context .

- Set is_mentioned to false if no exact dataset name/alias is present (flag terms only are not sufficient) .

Format your response as JSON array (include dataset_usage_status per item as one of: “none”, “mentioned”, “analyzed”):

[

{{

“dataset_name”: “ < dataset_name > “ ,

“is_mentioned”: true/false ,

“is_used”: true/false ,

“text_quote”: “ < exact quote from paper > “ ,

“reasoning”: “ < brief explanation as research analyst > “ ,

“context”: “ < surrounding text for context > “ ,

“dataset_usage_status”: “ < none | mentioned | analyzed > “ ,

“doi”: “ < dataset DOI if found, otherwise null > “

}}

]

Finding additional food security datasets

Two additional steps were applied to identify food security datasets outside of the eleven named datasets, working from the corpus of food-security publications identified in Step 3.

Finding potential additional datasets (Step 4): The LLM Discovery Prompt was applied to the full text of each of the publications validated as using a named food security dataset to extract any additional dataset names mentioned within the text. Unlike the Assessment Prompt, it operated without a predefined list of target datasets. The LLM was instructed to identify all dataset mentions, both from the named datasets and as novel candidates. The Discovery prompt was engineered as follows:

You are a specialized research analyst and expert at identifying datasets mentioned in academic publications. Your expertise lies in distinguishing between mere mentions and actual dataset utilization in research papers .

CONTEXT:

- You are analyzing a publication about: {publication_context}

- Known datasets in our database: {known_datasets_list}

- Text to analyze: {text_content}

VALIDATION CRITERIA:

A dataset is GENUINELY USED if:

1. The paper performs analysis, modeling, or experiments on the dataset

2. Results, findings, or insights are derived from the dataset

3. The dataset is central to the research methodology

4. Specific characteristics, variables, or samples from the dataset are discussed

5. The dataset is used for data processing, statistical analysis, or machine learning

6. The research methodology explicitly describes using this dataset

A dataset is ONLY MENTIONED (not genuinely used) if:

- It’s cited in related work or literature review without being used in the current study

- It’s mentioned as an example, comparison, or reference for future work

- It’s part of a background discussion without analysis

- It’s listed as a potential data source but not actually utilized

- It’s mentioned in acknowledgments or funding sections only

YOUR ROLE AS RESEARCH ANALYST:

As a specialized research analyst, you must carefully examine the context of each dataset mention to determine if it represents actual usage in the research methodology, analysis, or results. Consider the publication’s structure (methods, results, discussion) and the specific role each dataset plays in the research .

TASK:

1. Identify ALL dataset mentions in the text, including:

- Datasets that match known datasets in our database

- Completely new datasets not in our database

- Datasets mentioned in different formats or variations

2. For each dataset mention, provide:

- Dataset name (normalized)

- is_mentioned (true/false):

- whether the dataset’s exact name or alias appears explicitly in the text

- is_used (true/false):

- whether the dataset is actively used/analyzed in the research methods or results

- Exact text quote showing usage (if found)

- Context where it was mentioned

- Brief description if available

- Whether it matches a known dataset or is new

3. Focus on identifying:

- Dataset names in quotes or italics

- Acronyms that might be datasets

- Dataset names in references or citations

- Datasets mentioned in methodology sections

- Datasets in data availability statements

STRICT RULES:

- Set is_used to true ONLY when there is DIRECT EVIDENCE in the text_quote or context that explicitly cites the dataset by its exact name or one of its aliases AND indicates it was actually used in methods/analysis/results.

- Set is_used to false if the dataset appears only via flag terms (e.g., organization names like “USDA”, “NASS”) or generic references without explicit dataset name/alias.

- Set is_mentioned to true ONLY when the dataset’s exact name or one of its aliases appears explicitly in the text_quote or context.

- Set is_mentioned to false if no exact dataset name/alias is present (flag terms only are not sufficient).

VERY IMPORTANT:

- Do not add line breaks within the JSON text fields.

- Do not add any other text or comments outside the JSON text fields.

OUTPUT FORMAT (JSON):

{{

“discovered_datasets”: [

{{

“name”: “Dataset Name” ,

“confidence_score”: 8.5 ,

“is_mentioned”: true ,

“is_used”: true ,

“text_quote”: “Exact quote from paper showing usage” ,

“context”: “Full context where dataset was mentioned” ,

“description”: “Brief description if available” ,

“source”: “Source information if mentioned” ,

“domain”: “Domain/field if mentioned” ,

“size”: “Size information if mentioned” ,

“format”: “Data format if mentioned” ,

“access_info”: “Access information if mentioned” ,

“is_known_dataset”: true/false ,

“matched_known_dataset”:

“Known dataset name if matched” ,

“discovery_reasoning”: “Brief explanation as research analyst including mention vs usage assessment”

}}

] ,

“analysis_summary”: {{

“total_datasets_found”: 5 ,

“known_datasets_matched”: 3 ,

“unknown_datasets_found”: 2 ,

“confidence_distribution”:

“High confidence: 3, Medium: 1, Low: 1”

}}

}}

IMPORTANT:

- Be thorough but accurate - only include items that are clearly datasets

- Provide specific context quotes and exact text quotes when possible

- Distinguish between dataset mentions and general data references

- Include detailed reasoning for each discovery explaining mention vs usage assessment

CONTEXT:

- You are analyzing a publication about: {publication_context}

- Known datasets in our database: {known_datasets_list}

- Text to analyze: {text_content}

Because the raw output contained spelling variations and semantically similar dataset names used by different researchers to describe the same dataset, a string-similarity grouping procedure clustered the extracted names into candidate dataset labels. This reduction reflects the consolidation of name variants – for example, ‘ACS’, ‘American Community Survey’, ‘ACS Public Use Microdata Sample’ are three separate extractions that resolve to a single candidate label.

The specific approach was to apply a string-similarity grouping procedure using the RapidFuzz library’s Levenshtein ratio to cluster comparable extracted dataset names. First, extracted names corresponding to the eleven named datasets were removed. Pairwise similarity was then computed across the remaining names, and a greedy clique-based algorithm grouped names for which all members exceeded a 70% similarity threshold. Canonical labels were selected based on the highest average confidence scores.

The resulting candidate labels were then reviewed to produce a set of unique additional datasets. These were evaluated using two criteria applied sequentially.

First, the frequency of dataset appearance across the publications was used to prioritize names that appeared consistently across multiple papers rather than in isolated references.

Second, a food security domain expert confirmed that each remaining label referred to a distinct, accessible data source, excluding labels that resolved to survey questions, composite indices, or phrases too generic to identify a specific source. After deduplication of aliases and variant releases referring to the same underlying source, this process yielded a set of unique additional datasets as described in the Data Availability section.

The LLM also found that datasets were combined within individual studies. Information about these joins is beyond the scope of this paper, although joins of multiple datasets are often used answer complex questions about food systems⁴⁸.

Finding food security publications (Step 5) Mirroring Steps 1 and 2, a string search was deployed on the full Dimensions publications corpus for each additional dataset. Searches used the Dimensions DSL and were restricted to English-language, peer-reviewed publications from 2015–2025 with at least one U.S.-affiliated author. A food security concept filter was then applied to this broader set to retain only publications relevant to food security topics.

The resulting publication-dataset pairs from Steps 3 and 5 form the basis for the data-usage descriptors reported in the Results section⁴¹.

Results

LLM Workflow

The results of the workflow are described below.

Step 1: The string-search step for the eleven named datasets identified 11,915 publications mentioning at least one named dataset. These included both substantive uses and incidental references, which subsequent steps were designed to distinguish.

Step 2: The 11,915 publications were filtered to those relevant to food security using Dimensions’ concept relevance scores for “food security” and “food insecurity.” At any relevance score, 1,604 publications were associated with at least one of these concepts. However, many of these publications are only loosely linked to food security, as reflected in their relevance scores (Table 1). Publications with a food security relevance score greater than 0.6 were retained, yielding 1,019 unique publications. Because a single publication may mention more than one of the eleven named datasets, this corresponded to 1,114 publication-dataset pairs.

Table 1.

Frequency table of number of publications by food security concept relevance score.

Concept Relevance Score Threshold	Count	Percentage
>=0.1	1,604	100.00%
>=0.2	1,603	99.94%
>=0.3	1,600	99.75%
>=0.4	1,500	93.52%
>=0.5	1,357	84.60%
>=0.6	1,019	63.53%
>=0.7	488	30.42%
>=0.8	13	0.81%
>=0.9	0	0.00%
>=1.0	0	0.00%

Step 3: The 1,114 publication-dataset pairs were reviewed by the dataset review agent using the Assessment Prompt to determine whether each matched dataset was mentioned and substantively used. At the publication level, 807 unique publications contained at least one named dataset. Of these, 482 represented publications with high confidence of substantive dataset use. The full identification and screening process of Steps 1–3 is summarized in Fig. 2.

Fig. 2.

PRISMA diagram depicting identification of validated publications mentioning researcher-named datasets.

Step 4: The dataset discovery agent was applied to the full text of the 807 mention-validated publications to extract any additional dataset names. The 3,297 extracted dataset-name mentions were consolidated into 77 candidate dataset labels using a string-similarity grouping procedure, reflecting the consolidation of spelling variations and semantically similar names. For example, “ACS,” “American Community Survey,” and “ACS Public Use Microdata Sample” are three separate extractions that resolve to a single candidate label.

These 77 candidate labels were then evaluated using three criteria applied sequentially. First, the binary mention flag filtered out low-signal extractions. Second, frequency of appearance across the 807 publications prioritized labels that appeared consistently across multiple papers rather than in isolated references. Third, a food security domain expert confirmed that each remaining label referred to a distinct, accessible data source, excluding labels that resolved to survey questions, composite indices, or phrases too generic to identify a specific source. After deduplication of aliases and variant releases referring to the same underlying source, 16 unique additional datasets were retained.

Step 5: A string search was deployed on the full Dimensions corpus for each of the 16 additional datasets, using the same restrictions as Step 1 (English-language, peer-reviewed publications from 2015–2025 with at least one U.S.-affiliated author). Because each dataset was searched independently, results were deduplicated across datasets using Dimensions publication identifiers, yielding 147,402 unique publications. The same food security concept filter from Step 2 was then applied, retaining 1,063 unique concept-filtered publications. Authors, institutions, and journals were likewise deduplicated using ORCIDs, RORs, and ISSNs respectively.

Table 2 provides a detailed accounting of records of the workflow.

Table 2.

Publication and dataset counts across workflow steps.

STEP	LABEL	UNIT OF ANALYSIS	COUNT
1	Publications in Dimensions corpus	Publications	159,000,000
1	Removed before screening (year, language, affiliation)	Publications	151,526,945
1	Publications screened after basic filters	Publications	7,473,055
1	Publications mentioning at least one named dataset	Publications	11,915
2	Publications excluded by food security relevance filter	Publications	10,896
2	Concept-filtered publications retained	Unique publications	1,019
3	Publication-dataset pairs reviewed by LLM	Publication-dataset pairs	1,114
3	Publications excluded after mention validation	Unique publications	155
3	Publications with dataset mentioned and validated	Unique publications	807
3	Publications with data used and validated	Unique publications	482
4	Extracted dataset-name mentions from Step 3 corpus	Extracted dataset-name mentions	3,297
4	Clustered candidate dataset labels	Candidate dataset labels	77
4	Retained unique additional datasets	Unique datasets	16
5	Unique publications mentioning at least one additional dataset	Unique Publications	147,402
5	Concept-filtered publications retained	Unique publications	1,063

Data descriptor results

The goal of the workflow was to produce a set of data-usage descriptors.

Researcher named datasets

The eleven researcher-named datasets were validated as mentioned in 807 publications spanning 308 journals, with 2,920 authors affiliated with 645 institutions, accounting for 18,174 citations. The same metadata were validated as used in 482 publications that cover 205 journals, with 1,914 authors affiliated with 454 institutions, with 9,921 citations at the time the publication data were collected.

Additional datasets

The 16 additional datasets found through Steps 4–5 identified 1,063 unique food-security publications across 433 journals. These publications had 4,431 authors, affiliated with 834 institutions and had accumulated 21,833 citations at the time the data were collected. 738 of these publications represented food security research using datasets other than the initially named datasets. The search for food security research using the additional datasets contributed 3,350 previously unidentified authors, 475 additional institutional affiliations, and 10,342 additional citations beyond those identified through the named dataset search.

Thus, the additional dataset searches reveal a food security data user community that is substantially larger and more institutionally diverse than the named dataset search alone would suggest. While finding a broader set of food security datasets is beyond the scope of this paper, the process shows the scope for finding additional datasets via “community in the loop” LLM driven methodologies. Further iterations of the search loop and expanded human input would almost certainly reveal more datasets and users, thereby building the knowledge infrastructure around data use, and potential reuse.

Validation

Validation of LLM Results

A common validation approach is to benchmark the output of an LLM relative to a gold standard. In this case, there is no gold standard corpus that unambiguously identifies dataset mentions and uses in a set of publications. There are well known problems associated with generating such a gold standard with human annotators, particularly when the categorization task is complex⁴⁹. As Plank notes, “human label variation” is common⁵⁰ and very often reflect fundamental and reasonable disagreements on what is a reasonable categorization.

The central question in the validation exercise is one of scalability and plausibility: does the LLM, given the same instructions as a domain expert, classify dataset mentions and uses in publications at a level comparable to human judgment? If the LLM can perform this classification task with the same consistency as a domain expert at a fraction of the cost, then there is the potential for the LLM based approach to scale. A number of papers have argued that inter-rater agreement is a robust metric^51,52. We also report plausibility, the fraction of cases in which the LLM agrees with at least one human rater; this measure has become more common in evaluating LLM alignment⁵³.

The validation exercise focuses on the Household Food Security Survey Module for three reasons. First, it is the focal point of this paper, although other datasets are of interest as well. Second, it appears most frequently and in the widest variety of contexts across the food security literature⁵⁴. Finally, since we were interested in describing the process for comparing LLM output with the judgment of humans, we thought it best to minimize the noise that would be introduced by including other datasets.

Fifty publication-dataset pairs involving HFSSM were drawn from the researcher-named dataset pipeline (Steps 1–3) and independently reviewed. Each reviewer evaluated, for each pair, whether the HFSSM was mentioned in the publication and whether it was substantively used in the analysis, following the same prompt given to the LLM and working from the full text of the publication.

Reviewers and the LLM applied the same binary logic: a dataset is classified as mentioned if its name or a recognized alias appears explicitly in the text, and as used if there is direct textual evidence of substantive engagement with the dataset in the research. The survey questions derived from HFSSM in a study’s own instrument design, for example, was treated as a use, consistent with the prompt criterion that “specific characteristics, variables, or samples from the dataset are discussed.” However, a dataset appearing only in a background discussion, literature review, or reference list was classified as mentioned, but not used. The result is a direct comparison of classifications on the same judgment task.

Table 3 reports pairwise agreement across all four raters (the three human reviewers and the LLM), for both mention and use.

Table 3.

Agreement Matrix for Mention and Use.

Metric	Mention (%)	Use (%)
Concurrence Rate (LLM vs Human Majority)	76.0	56.0
Plausibility Rate (LLM vs Minimum 1 Rater)	80.0	82.0
Pairwise Agreement - Raters 1 and 2	98.0	68.0
Pairwise Agreement - Raters 1 and 3	94.0	74.0
Pairwise Agreement - Raters 2 and 3	92.0	74.0
Overall Human Pairwise Agreement	94.7	72.0

Note: Concurrence rate is the proportion of publication-dataset pairs in which the LLM classification matched the human majority vote (yes or no for mention; yes or no for use). Plausibility rate is defined as agreement with at least one of the three reviewers.

In the case of mentions, human reviewers agreed with each other 94.7% of cases on average. The LLM agreed with at least one human reviewer 80% of the time. We reviewed the cases where the LLM classified the publication as not mentioned while all three human reviewers classified it as mentioned. In most cases the humans were correct: the name of the dataset appeared in the text. For others, disagreement stems from variation in how the dataset name appeared in the text and where it appeared, such as in a table rather than in the body of the article.

In the case of use, human reviewers agreed with each other 72% of the time. The LLM agreed with at least one reviewer 82% of the time. It is worth noting that the LLM applied a stricter interpretation of use than human reviewers, particularly in cases where a study used an instrument derived from the HFSSM rather than the HFSSM directly. Lindahl51 has pointed out that disagreements across annotators can be due to differences in interpretation of boundaries and the appropriateness of labels. Three cases from the validation sample illustrate this pattern. The Brazilian Household Food Insecurity Measurement Scale (EBIA) adapts the HFSSM questions for use in Brazil; the Hunger Vital Signs is a two-question screening tool for clinical settings derived from the HFSSM; and the food security scale used in the Korean Welfare Panel Study (KOWEPS) adapts HFSSM questions for a Korean national survey context. In each case, the LLM and one human reviewer classified these as non-use of HFSSM while two of the human reviewers classified them as use. The linked dataset is therefore likely a conservative count of HFSSM uses rather than an exhaustive one.

Both the plausibility and concurrence rates are well within the ranges observed in related work^51,55. The results are certainly promising for the use of the LLM as a scalable tool for this classification task. It is worth noting that producing human ratings of data use is not likely to be feasible at scale. It took each of the three human raters about two hours each to read and score the 50 documents, even after the documents were selected. In another comparison, when the American Statistical Association attempted to quantify how much federal statistics were used, and did not have the funding for a probability survey, it sent a Qualtrics user solicitation to professional associations, requesting that it get forwarded to their members. That effort to capture use of federal statistical datasets resulted in about 350 usable responses, which is likely to be an undercount^56,57.

Implementation in the National Data Platform

The workflow presented is intentionally schema-agnostic at the discovery stage. For repository integration, however, the retained usage descriptors must be stabilized as metadata objects for all the identified datasets in the NDP. In this paper, we propose a minimal data-usage descriptor with the minimum set of fields to serve as a stable metadata object suitable for repository integration. The descriptor should enable indexing and search by external tools such as Dataset Search⁵⁸ to enable integration with other data platforms. The proposed descriptor is anchored at the Dataset entity. Each dataset aggregates reviews from individual publications, where each review records mention and use confidence scores using native schema.org types. No custom namespace is required. Figure 3 provides the schema overview; Fig. 4 provides how it would be applied for a specific example.

Fig. 3.

Schema Overview.

Fig. 4.

Example of Real Publication Record.

The food security data-usage descriptors, as well as the reproducible workflow for constructing data-usage descriptors presented in this paper, are being used to support a usage-based contextual data discovery service within the National Data Platform (NDP; https://nationaldataplatform.org/). The NDP is designed to enhance data accessibility and usability for researchers across diverse scientific domains by providing a federated, extensible data ecosystem of data resources and services. It provides a cohesive platform that supports comprehensive data services for ingestion, indexing, curation, and analysis.

In the case of food security, the NDP provides food security researchers – the data reuser constituency identified by Borgman and Groth, with a quick overview of how the data are used in the Food Security Dashboard⁵⁹ (Fig. 5), including links to the publications, citations, journals, authors, institutions, and the topics studied¹¹. The Dashboard also provides them with a sense of the intensity of the use – for example, there were over 13,000 citations to publications using the HFSSM in the period under study.

Fig. 5.

The HFSSM data usage statistics in the NDP.

The NDP could expand this understanding to many domains because it hosts diverse data from a range of sources, driven by their contributor base. These include domain data repositories (e.g., the NIH MIDRC Data Commons; https://data.midrc.org/), streaming data from instruments and observatories (e.g., data from the NSF EarthScope Consortium (https://www.earthscope.org/) and the NOAA and NASA GOES Satellite Network; https://science.nasa.gov/mission/goes/).

NDP also integrates end-to-end cyberinfrastructure for real-time, data-driven simulation, prediction, and visualization of wildfire behavior (WIFIRE), which can be used to model and predict wildfire spread²⁰. Moving forward, NDP will continue to expand its data resource to include a range of emerging data sources such as streaming data from Internet of Things networks, smart cities, low-orbit satellites, and social media, and support workflows that combine these data streams with more traditional data sources to support data-driven understanding and decision making. A major advantage is that because the workflow has been rigorously developed to be reproducible, it is relatively low-cost to implement and scalable to multiple domains.

Expansion to other domains

Although the focus of this paper has been on describing the process and methods for the use case of food security, the potential to develop usage-data descriptors to other domains has been successfully tested using the same workflow on datasets named by two disparate communities: workforce data (labor economics) and pediatric asthma data (health sciences) – for a different project. These data-usage descriptors developed for the two additional core datasets have already been hosted in the NDP (Workforce Dashboard⁶⁰; Pediatric Asthma Dashboard⁶¹).

The potential to generate the same metadata was confirmed. In particular, it is possible to compare the differences in the knowledge infrastructure for labor and workforce research for a well curated and funded dataset relative to data that are valuable but not developed for research purposes. In the most highly curated dataset – a major occupational dataset (called O*NET⁶²) funded by the Department of Labor – the data contractor identified 1,337 publications in list format for labor, industrial and vocation research for the period of 2015–2025. The search process documented in this paper, when applied to search for O*NET uses, found 2,290 publications, 6,511 authors, at 1,496 institutions. In less curated datasets, like unemployment insurance wage records which are produced for administrative, not research purposes⁶³, there is no single source of information about the use of these data. However, the search process found 1,148 publications with 2,309 users at 669 institutions.

While the NDP serves as the primary repository host for these descriptors, statistical harmonization platforms like Data Commons represent the ideal venue for their consumption⁶⁴. Integrating usage descriptors directly into the Data Commons Knowledge Graph (which natively utilizes schema.org schemas) would allow researchers querying harmonized variables (e.g., food insecurity rates) to instantly discover which publications, institutions, and co-occurring datasets are associated with those specific statistical series. This bridges the gap between raw statistical data access and qualitative scientific provenance. Furthermore, with NDP publishing this data in schema.org, makes it widely discoverable through tools such as Google Dataset Search⁵⁸.

As Data Commons, or users of Data Commons, build reasoning products on top of this repository, the community based risks and warnings ensure that the right footnotes are traveling with the data throughout the decision making process. Ensuring these footnotes are machine readable also makes them actionable by future agentic systems.

Discussion

The data curation workflow presented in this article shows both how data-usage descriptors can be developed and how the methodology can be used to learn about additional datasets and their usage, regardless of domain. The workflow, queries, and file structure can be reused to construct similar data-usage descriptors for other named datasets or research domains. In areas where core datasets are at risk of retirement or where the data landscape is fragmented, this approach provides a practical way to document data use, identify gaps, and locate existing expertise for future work on science and innovation.

There are at least three ways in which this work could advance the use of data in both science and innovation.

First, data-usage descriptors could be used to increase understanding of how datasets are used in scientific research areas and thus expand the knowledge infrastructure around data¹¹. As noted in the introduction, although only about 1% of datasets are directly cited once or more, the search for usage of the named datasets through publications identified thousands of citations that were hitherto unknown.

The approach could be expanded to go beyond the initial list of researcher named datasets to find additional datasets, and their usage, in the same field. Notably, in the food security case, the eleven researcher-named datasets were augmented by another 16, contributing 738 additional food-security publications beyond those already identified through the named dataset search (1,752 unique publications in total). Data creators who want to get more credit for their data being used could make their data more findable.

For fields affected by dataset retirements, the data-usage descriptors can point users to potential substitute data sources and ways of identifying the communities that already work with them. It can inform data funders about the extent of use when data are at risk of being retired. It can complement existing manual methods by providing less costly, more expansive, and less selective than manual approaches.

Second, the data-usage descriptors generated by this workflow could be used by science agencies, publishers, and research fields, to help reward and incentivize investments in data. They describe data inputs to a research fields by showing how often particular datasets appear in the literature, and which alternative datasets are drawn in from other domains, including labor statistics, public health data, or demographic data. They also characterize data outputs and the evolution of activity over time, including trends in publication counts, citations, and journal outlets associated with specific datasets or groups of datasets. The combined information could help both inform funding decisions and reward the production and reuse of important datasets by researchers¹⁵. Data funders interested in supporting research reproducibility, replication, and robustness^65–68 could use the approach to track, discover, reward, and proactively respond to changes in the research data ecosystem, just as they do for research papers. We hope that a systematic account of which datasets are in use, which communities depend on them, and what alternatives exist, would help document the consequences for empirical research of decisions like the one to eliminate the Household Food Security Survey Module.

Third, the production of data usage statistics can help support the development of an institutional framework that recognizes data as an output that propels scientific advances in the same way that research papers do²⁵. Data citations, metadata, classifications, persistence are all in the early stages of the formation of norms. Systems like the National Data Platform and Data Commons can be critical in providing institutional norms that enable data to exist in a persistently referenceable, navigable and contextual framework^20,22.

While the results are promising, much more can be done. The paper is illustrative in nature and represents a conservative view of the use of datasets in food security research that could be expanded in coverage and scope.

The validation exercise demonstrates the limits of the current approach and how the value could be increased with the engagement of the relevant research community. The prompts used follow a zero-shot approach, where the LLM receives a definition of mention and use and applies it without labeled examples. We saw that LLM agreed with at least one human rater in large majority of the cases. However, LLM’s concurrence with the majority could also be higher, if additional information were to be provided to the LLM with labeled examples of common corner cases before classification that could improve concurrence both among the raters and between LLM and the raters.

The research community could also help define how “use” should be defined in their domain. Agreeing on what constitutes the boundary between a use versus a mention of a particular dataset requires domain experts to work through cases, and to document those decisions so others can build on them. Researchers who apply this workflow to new datasets will encounter their own edge cases, resolve them through discussion, and in doing so contribute to a growing shared understanding of how datasets are used in their field. The workflow provides structure for that process.

Another way in which the current approach could be improved is through the enhancement of the search methods themselves. For instance, the characterization of “food security” as a Dimensions concept in the publication corpus itself could be revised by the food security community, potentially using the LLMs being developed by Digital Science to do so⁶⁹.

Much more could be done. For example, much could be learned by examining the join metadata which describe how different datasets are combined in individual studies. Established integration approaches could help researchers transition to alternative datasets when long-standing sources are reduced or retired⁷⁰. As another example,investing in compute infrastructure could make scaling to other domains and frequent updating feasible. The pipeline used on this paper ran on a single GPU with a local LLM, so additional iterations would require proportionally more processing time. With investment in compute infrastructure, it would be feasible to find more data by bootstrapping beyond the two stages described in this paper. The pipeline architecture itself is not the bottleneck.

The potential benefits extent beyond the academic realm. The work also provides a quantitative approach to documenting and responding to the consequences of dataset churn so that funders of scientific data, data repositories, and researchers can respond substantively to decisions like the elimination of the Household Food Security Survey Module.

We hope that this initial foray into developing data-usage descriptors, and their incorporation into the NDP and Data Commons, will stimulate additional engagement and adoption by the scientific community.

Appendix

Table 4, Table 5

Table 4.

File structure and variable definitions for named publication-dataset pairs.

Field name	Type	Description
publication_id	String	Unique identifier assigned to each publication within the Dimensions corpus.
publication_title	String	Full title of the publication.
publication_doi	String	Digital Object Identifier (DOI) for the publication.
validated_dataset_id	String	Internal identifier for the validated dataset entity.
validated_dataset_name	String	Canonical name of the dataset referenced in the publication.
is_mentioned	Boolean	Whether the dataset’s name or a recognized alias appears explicitly in the publication text (true/false).
is_used	Boolean	Whether the dataset was substantively used in the study’s analysis rather than only mentioned (true/false).

Table 5.

File structure and variable definitions for additional publication-dataset pairs.

Field name	Type	Description
publication_id	String	Unique identifier assigned to each publication within the Dimensions corpus.
publication_title	String	Full title of the publication.
publication_doi	String	Digital Object Identifier (DOI) for the publication.
new_name	String	Canonicalized name of the newly discovered dataset.
new_is_mentioned	Boolean	Whether the additional dataset’s name or alias appears explicitly in the publication text (true/false).
new_is_used	Boolean	Whether the additional dataset was substantively used in the study’s analysis (true/false).
new_description	String	Short description of the dataset as inferred from the publication context.
new_source	String	Organization or agency responsible for producing the dataset.
new_domain	String	Thematic or disciplinary domain of the newly identified dataset (for example, labor statistics, public health, or environmental monitoring). This reflects the dataset’s subject area rather than the publication’s research topic (for example, a food security study may use a dataset classified under the public health or labor domains).

Author contributions

L.C. contributed to the investigation, writing, reviewing and editing, visualization, validation, formal analysis, and project administration. J.L. contributed to the conceptualization, methodology, writing of the original draft, reviewing and editing, project administration, supervision and funding acquisition. R.L. contributed to the methodology, software, validation, formal analysis, investigation, data curation, writing – review and editing, and visualization. N.N. contributed to the methodology, conceptualization, formal analysis, investigation, writing – review and editing M.P. contributed to the conceptualization, design, implementation, validation, writing, and acquisition of funding. S.P. contributed to software, validation, resources, data curation, writing – reviewing and editing, project administration, supervision. All authors reviewed the manuscript.

Funding

This work was supported by gifts, grants and contracts from the US Department of Agriculture 58-5000-8-0053, the National Science Foundation contract 49100422C0028, grant 2333609, grant 2440195, Schmidt Sciences, Patrick J. McGovern Foundation, Overdeck Family Foundation, the Alfred P. Sloan Foundation, and with the support of Google.org. Digital Science provided financial support to Rafael Ladislau.

Data availability

The data generated by this study are publicly available at Zenodo (10.5281/zenodo.19478418). The repository contains primary data files, supporting files (counts, aliases, and dictionaries), and JSON-LD descriptors.

Code availability

The complete set of data-usage descriptors, supporting data files, and reproducible code are publicly available⁴¹.

The pipeline is running across datasets for three domains: food security, pediatric asthma, and workforce, and these are all hosted on the National Data Platform. Across these domains, the system currently manages more than 509,000 indexed publications, 5,613 completed LLM analyses, 107 tracked datasets, 1.49 million author records, and 26,591 institution records. The system itself is built using a queue-based architecture with RabbitMQ workers, which means it can scale by increasing the number of worker replicas rather than redesigning the workflow. In its current deployment, the system reached a peak throughput of 1,112 analyses in a single day.

The accompanying documentation describes field definitions for the datasets used in the food security research.

File A: Named Publication-Dataset Pairs

fileA_usda_publication_dataset_pairs.csv is the primary data file generated through Step 3 of the workflow. Each record represents a verified instance where a dataset is mentioned within a publication.

Publications mentioning datasets were first detected using a string search approach with the Dimensions API, which identifies dataset references within the full text. These initial matches were then validated through the LLM-assisted workflow. Validation fields indicate whether the dataset was simply mentioned or used for analysis, along with the supporting text excerpt and binary mention and use classifications.

All variables use standardized naming conventions and controlled vocabularies. Dataset names are harmonized to match canonical titles (for example, “Agricultural Resource Management Survey” instead of “ARMS”). Text fields preserve the original context to facilitate reproducibility and secondary analyses. Table 4 summarizes the file structure and variable definitions.

File B: Publication-Dataset Pairs Using Alternative Datasets

fileB_additional_datasets_publication_pairs.csv documents newly identified datasets that were not included in the initial list of named datasets but were detected through the LLM. Each record represents a verified instance where a previously unlisted dataset was mentioned or used within a peer-reviewed publication. The file extends the dataset inventory by capturing additional data sources that appear in the same research domain.

Each record links a publication to a newly discovered dataset, along with metadata describing the dataset’s name, context of use, and LLM-generated confidence scores for both identification and substantive usage. The file also provides contextual text excerpts, short dataset descriptions, and classification fields that indicate the dataset’s domain and source organization.

All dataset names are harmonized where possible to canonical titles, and metadata fields preserve the original text context to support reproducibility and follow-up verification. Table 5 summarizes the file structure and variable definitions. 

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41597-026-07753-8.