A dataset of neural network architectures generated via large language models

Abstract

Neural networks are increasingly used to support decision-making. To verify their reliability and adaptability, researchers and practitioners have proposed a variety of tools and methods for tasks such as NN code verification, refactoring, and migration. These tools play a crucial role in guaranteeing both the correctness and maintainability of neural network architectures, helping to prevent implementation errors, simplify model updates, and ensure that complex networks can be reliably extended and reused. Yet, assessing their effectiveness remains challenging due to the lack of publicly available diverse datasets of neural networks that would allow systematic evaluation. To address this gap, we leverage large language models (LLMs) to automatically generate a dataset of neural networks that can serve as a benchmark for validation. The dataset is designed to cover diverse layer types and to handle multiple input data types and tasks. In total, 608 samples are generated, each conforming to a set of precise design choices. To further ensure their consistency, we validate the correctness of the generated networks using static analysis and symbolic tracing. We make the dataset publicly available to support the community in advancing research on neural network reliability and adaptability.

Background & Summary

Neural networks (NNs) have become an integral component of modern software systems, enabling them to perform complex prediction and decision-making tasks. Their widespread adoption spans various domains and applications, and they are increasingly at the core of software development. To support the design and development of NNs, a variety of tools have been proposed to ensure their reliability and adaptability. Verification has become a key research area, ensuring that models behave as expected and satisfy critical properties of safety or robustness¹. In parallel, refactoring techniques have emerged to enhance models efficiency and structure without severely altering their behaviour². Given the rapid evolution of NN technologies, several tools have been developed to facilitate NN migration either by transferring trained, ready-to-use models or by migrating the NN code itself across frameworks^3,4.

Existing NN-related datasets are primarily designed for architecture benchmarking or to support NN training by providing samples for prediction tasks^5,6. These datasets are therefore not suitable for thoroughly evaluating tools that target NN verification, refactoring, and migration, which require a diverse set of NN architectures covering different designs, layers, tasks, and complexity levels. Evaluating such tools on a few architectures does not allow a comprehensive assessment as it does not expose the full range of NN designs they may encounter in practice. To properly evaluate the robustness and generality of NN tools, a diverse dataset of architectures is needed. However, manually collecting or designing such models is time-consuming and does not guarantee representativeness. This limitation slows progress and hinders fair and consistent evaluation across NN tools.

Large language models (LLMs) have revolutionized various domains, ranging from natural language understanding to software development and scientific computing⁷. Besides their traditional applications, LLMs have shown great potential in generating synthetic datasets. For example, they have been used to generate tabular data samples to support NN training⁸. Moreover, LLMs have shown promising results in code generation, producing functional code based on detailed user instructions⁹. Given these capabilities, LLMs stand as a promising tool for creating diverse and high-quality code-related datasets that could help overcome existing limitations in evaluation and data availability.

In this paper, we propose a dataset of NN architectures generated using LLMs. Specifically, we rely on GPT-5¹⁰ to create a diverse collection of NN models designed to support research on NN reliability and adaptability. Carefully crafted instructions were used to guide the generation to ensure diversity in architecture types, tasks, input types, and complexity levels. The generated NNs can handle various input data types and perform a range of prediction tasks. Moreover, they are designed with different complexity levels to better reflect the variability of design that generally exist in NN models. In total, 608 NNs were generated, representing, to the best of our knowledge, the first dataset of NN architectures created to facilitate research on NN reliability and adaptability. We also perform static analysis and symbolic tracing to ensure the validity of the architectures and that they adhere to the instructions in the prompt. We make our dataset publicly available at Zenodo¹¹ to support the community and encourage further research on NN development.

Methods

Our NN dataset¹¹ was created in three main steps: Requirements Definition, Prompt Construction, and LLM-based Generation, which are illustrated in Fig. 1.

Fig. 1.

Overview of our LLM-driven NN dataset generation.

Requirements definition

The first step in the process is to define the characteristics of the NN architectures to be generated. Since LLMs need specific instructions to guide the generation, it is important to feed them with precise requirements for the desired NN architectures. Specifically, we define four categories of requirements to ensure diversity:

Architecture

It refers to the type of layers used in the NN definition. Seven categories are considered: MLP, CNN-1D, CNN-2D, CNN-3D, RNN-Simple, RNN-LSTM, and RNN-GRU. These categories are chosen to guarantee that our dataset is diverse and covers the fundamental layers in NNs. We note that each of these architecture types is characterised by the use of a specific NN layer, hereafter referred to as a characterising layer (CL), as presented in the first two columns of Table 1.

Table 1.

CL and complexity thresholds of NN architectures.

Architecture		Complexity
Type	CL	Width Thr	Width Variable	Depth Thr
MLP	Linear	$min\mathrm{(128,}U)$	$out\_features$	4
CNN-1D	Conv1d	$min\mathrm{(128,}U)$	$out\_channels$	4
CNN-2D	Conv2d	$min\mathrm{(8,}W\mathrm{//8)}$	$out\_channels$	4
CNN-3D	Conv3d	$min\mathrm{(8,}W\mathrm{//8)}$	$out\_channels$	4
RNN-Simple	RNN	$min\mathrm{(128,}U)$	$hidden\_size$	2
RNN-LSTM	LSTM	$min\mathrm{(128,}U)$	$hidden\_size$	2
RNN-GRU	GRU	$min\mathrm{(128,}U)$	$hidden\_size$	2

Thr: Threshold; U: upper_bound_features; W: image_width.

Task

It refers to the prediction task performed by the NN. We considered four main tasks to make the NNs diverse: Binary-classification, Multiclass-classification, Regression and Representation-learning. We note that we excluded Multilabel-classification as it closely resembles Binary-classification: both use a sigmoid output (explicitly or via losses like BCEWithLogitsLoss), differing mainly in the number of output neurons. Hence, including it would not contribute to the diversity of the dataset.

Input type and scale

NNs can learn from a variety of input data types, each having unique characteristics and scales. These differences affect the learning process and determine which architectural choices are most effective for capturing the underlying structure of the data. Besides input type, we have also selected specific scales or ranges for input to drive the generation of NNs with diverse architectures. Changes in the dimensionality of input data affects how layers are structured and connected, and can also influence the inclusion of components such as normalization or activation layers, resulting in NNs with varied layer composition. We considered four input data types, Tabular, Time Series, Text, and Image, each with its corresponding input scale. These types and scales are summarized in columns 1 and 3 of Table 2.

Table 2.

Combinations of NN requirements considered for dataset generation.

Input Type	Possible combinations
	Arch	Input Scale	T × C	Total
Tabular	MLP	$f\mathrm{<50,}f\mathrm{>2000}$ (2 scales)	4 × 4	32
Time series	CNN-1D, RNN-Simple, RNN-LSTM, RNN-GRU	$s\mathrm{<50}$ $\&$ $f\mathrm{<50,}$ $s\mathrm{<50}$ $\&$ $f\mathrm{>2000,}$ $s\mathrm{>2000}$ $\&$ $f\mathrm{<50,}$ $s\mathrm{>2000}$ $\&$ $f\mathrm{>2000}$ (4 scales)	4 × 4	256
Text	CNN-1D, RNN-Simple, RNN-LSTM, RNN-GRU	$s\mathrm{<50}$ $\&$ $v\mathrm{<1}k,$ $s\mathrm{<50}$ $\&$ $v\mathrm{>100}k,$ $s\mathrm{>2000}$ $\&$ $v\mathrm{<1}k,$$s\mathrm{>2000}$ $\&$ $v\mathrm{>100}k$ (4 scales)	4 × 4	256
Image	CNN-2D, CNN-3D	$r\mathrm{<64,}r\mathrm{>1024}$ (2 scales)	4 × 4	64

Arch: Architecture; T: Task; C: Complexity.

f: features; s: seq_length; v: vocab_size; r: resolution.

Complexity

It influences the number of layers and the width of NNs, and is used to guide the generation of networks ranging from simple to more advanced structures. The complexity has been tailored to each architecture type, reflecting its distinct requirements for input data and intended tasks. We define two thresholds: one for the number of CLs and one for the width of the first CL.

Table 1 summarizes these width and depth thresholds, along with the width variable, for the different architecture types. In total, four complexity levels were considered: (1) Simple, referring to an NN with a basic architecture, where the number of CLs and the width of the first CL (c.f., Table 1) are expected to be below the thresholds; (2) Wide, an NN where the width of the first CL is at least the width threshold; (3) Deep, an NN whose total number of CLs is at least the depth threshold; and (4) Wide-Deep, an NN whose number of CLs and width of the first CL are at least the respective thresholds. For example, in a simple MLP, the number of Linear layers does not exceed four and the first Linear layer contains up to min(128, upper_bound_of_features) output neurons. Here, upper_bound_of_features is either 50 or 2000, as defined by Input type and scale requirement of tabular data.

For CNN-2D and CNN-3D, the width of the first CL is defined as $min\mathrm{(8,}W\mathrm{//8)}$, where $W$ is the image width. The value of 8 was chosen to ensure that simple NNs remain relatively small even for large images, while wide NNs still exceed this baseline for smaller images. For MLP, CNN-1D, and RNNs (i.e., RNN-Simple, RNN-LSTM and RNN-GRU) architectures, the first CL width is defined as $min\mathrm{(128,}U)$, with $U$ the number of input features. 128 was chosen as a round, sufficiently large value to allow wide NNs to scale with the input while keeping simple NNs relatively small. These specific values were selected to systematically preserve the relative distinction between simple and wide NNs across varying input sizes.

For Text input type, CNN-1D and RNNs architectures can be used. However, since the input is text, the network starts with an Embedding layer rather than the standard CL of the architecture. The width threshold is therefore applied to this Embedding layer. Specifically, the width variable corresponds to embedding_dim of the Embedding layer, with a threshold of $min\mathrm{(128,}upper\_bound\_of\_vocab\_size)$.

To avoid ambiguity in complexity types, we add the precise definition to the complexity level based on both the architecture type and the input type and scale. An example of the full complexity definition is shown in Step 1 of Fig. 1.

Listing 1: LLM prompt template.

Prompt construction

Generating NN code with an LLM requires a carefully crafted prompt. We base our instructions on the requirements outlined in the previous section, providing details about the network architecture, input type and scale, prediction task and complexity. This approach ensures that the generated NN code follows the specified requirements, producing a variety of NNs that collectively cover a wide range of architectural characteristics and input data configurations. We present in Listing 1 our prompt template.

In the template, Architecture, Learning Task, Input Type + Scale, and Complexity are variables that change with each generation. We also provide additional instructions to the LLM, such as ensuring that the generated code is fully self-contained, avoiding the use of parameters defined outside the network, which could otherwise introduce unnecessary dependencies. We further avoid introducing custom block classes or helper functions, ensuring the generated networks are expressed in the standard way supported by the framework, keeping the dataset structured and consistent.

LLM-based generation

We employ GPT-5¹⁰ to produce the NN code. We use the requirements defined in Requirements Definition section to populate the prompt template presented in Prompt Construction section. To that end, we first determine the possible combinations of requirements as some combinations are irrelevant or do not correspond to valid NN architectures. For example, using an MLP to process image inputs is not a typical design choice, as it does not leverage the spatial structure of the data. We present a summary of the possible combinations of requirements in Table 2. In total, our dataset contains 608 NNs generated by GPT-5.

Data Record

The dataset is publicly available on Zenodo¹¹ under a CC BY 4.0 license. It consists of 608 Python files, each representing a single neural network architecture. The files are stored in a flat directory with no subdirectories.

File naming convention

Each file in the dataset follows the format: architecture_task_input-type-scale_complexity.py. Each component takes one of the following values:

• architecture: one of seven types: MLP, CNN-1D, CNN-2D, CNN-3D, RNN-Simple, RNN-LSTM, RNN-GRU, corresponding to the characterising layers Linear, Conv1d, Conv2d, Conv3d, RNN, LSTM, and GRU respectively.

• task: one of four prediction tasks: binary-classification, multiclass-classification, regression, representation-learning.

• input-type-scale: combines the input data type, one of tabular, time series, text, or image, with its scale, small or large, indicating the scale of features, sequence length, vocabulary size, or resolution depending on the input type.

• complexity: one of four levels: simple, wide, deep, wide-deep, reflecting the depth and width of the network.

For example, mlp_classification-binary_tabular-large_simple.py refers to a simple MLP for binary classification on large tabular input with a simple complexity level.

File contents

Each .py file contains a comment at the top describing the exact prompt used to generate the network, followed by a single Python class inheriting from torch.nn.Module and defining the neural network, including the forward() method defining the computation graph.

Data Overview

Diversity across architecture types, tasks, input types, and complexity levels is a key aspect of our dataset¹¹. While the NN requirements specified for the generation define certain NN characteristics, such as the architecture type, the exact layers and tensorops used to construct the NNs are purely determined by the LLM. We therefore inspected the NN components in our dataset and found a total of 6842 layers and tensorops, comprising 38 unique types, which confirms the diversity of the structural components.

In the complexity requirements, we explicitly defined deep NNs as those with a number of CLs above a certain threshold. While this threshold sets the minimum number of CLs, the exact number of CLs, as well as all other layers, is determined by the LLM. We examine the distribution of NN depth in Fig. 2, which presents boxplots showing the depth of all layers and of CLs for both the full dataset and for NNs grouped by complexity. The overall depth of NNs ranges from 2 to 35. For NNs grouped by complexity, the patterns are consistent: Simple NNs generally have smaller depth (both for all layers and CLs), followed by Wide, Deep, and Wide-Deep NNs. For Wide NNs, no depth threshold was set; only the width was constrained. Nevertheless, the LLM produced slightly deeper NNs compared to Simple NNs, which could be explained by the fact that Wide NNs require more layers to perform feature transformations. Overall, the NN dataset contains a variety of layers and tensorops and includes NNs of different depths.

Fig. 2.

Depth of all layers and of CLs for the full NN dataset and for NNs grouped by complexity level.

Technical Validation

The validity and quality of our dataset was confirmed through compliance verification, functional validation, and a utility demonstration on a use case of a NN migration tool.

Compliance verification

Each of the 608 neural network architectures was automatically verified against the four requirements defined during dataset construction: architecture type, task, input type and scale, and complexity.

To verify the architecture requirement, the presence of at least one characterising layer (CL) was confirmed for each architecture. For the task requirement, the output layer configuration was inspected to ensure it matches the expected task, with the appropriate number of output neurons for tasks such as classification and regression. For the input type and scale, the first CL was inspected to verify that its parameters match the expected specification. This includes checking parameters such as in_features for tabular data, and in_channels or input_size for time series. For complexity, the number of CLs and their parameters were compared against the thresholds specified in Table 1.

This verification combines static analysis using AST¹², which parses the NN source code to extract all layers including functional calls¹³, and symbolic tracing to confirm structural integrity and correct execution under expected inputs.

To verify the correctness of the validation process itself, we selected two neural networks per architecture type and manually inspected each one against the four defined requirements to confirm their correctness. We then ran the validation process on these 14 NNs and confirmed that they were reported as compliant with the requirements. Furthermore, we manually introduced targeted errors into these NN architectures and confirmed that they were all correctly flagged as non-compliant.

In total, 8 out of 608 architectures were found non-compliant, all belonging to the RNN-LSTM and RNN-GRU categories with large time series input features. The common issue was the use of a Linear projection layer before the CL (LSTM or GRU), which results in breaking the sequential structure of the input data and preventing the temporal modelling. These 8 architectures were regenerated and successfully verified, resulting in a fully compliant dataset of 608 architectures.

Functional validation

To confirm that the generated code is executable and functionally correct, one architecture per input data type was selected and trained on standard benchmark datasets^14–17. The results showed consistent performance, confirming the correctness and usability of the NN generated architectures.

Utility demonstration

To further demonstrate the practical utility of the dataset, we applied it to evaluate a tool for neural network migration³. While the tool correctly migrates standard NN architectures, the dataset’s variety and structural richness exposed varied PyTorch coding patterns that the tool did not initially support. The initial execution of the tool on the dataset showed that 200 out of 608 architectures were successfully migrated. Guided by the variety of patterns found in the dataset, the migration tool was extended to support a broader range of NN designs. As a result, all 608 architectures were successfully migrated, confirming the dataset’s utility as a benchmark and its ability to benefit NN tools and drive their robustness and generalisation.

Related Work

In this section, we review related works and describe the originality of our NN dataset compared to existing datasets.

Neural network datasets

Existing datasets related to NNs generally focus on Neural Architecture Search (NAS)¹⁸, which aims to automate the process of NN design to achieve the highest performance. NAS-Bench-101¹⁹ has proposed a set of CNN architectures mapped to their performance metrics. Similarly, NAS-HPO-Bench²⁰ and NAS-HPO-Bench-II²¹ have considered the problem of joint optimization. Recently, Younger²² has been proposed as a dataset containing NN architectures represented as directed acyclic graphs to assist in NN design and optimization. Other traditional datasets have been also widely used to train NNs for a range of different tasks^15,23–27.

Our work differs from these approaches by providing a dataset of diverse NNs designed to meet specific requirements on architecture, task, input, and complexity. This diversity makes it suitable for NN reliability and adaptability methods such as NN verification^1,28, refactoring^2,29, and migration^3,4,30. In contrast, the above approaches focus on NN training or architecture search, which would prioritise performance and limit the diversity of the NNs.

Mining software repositories³¹ can be considered as an alternative approach to create a dataset of neural networks. However, mined NN code potentially depends on external components and custom files, making it not directly usable without significant preprocessing and cleaning effort and does not guarantee a balanced and representative coverage of the design space.

LLM-based dataset generation

LLMs have been used for NAS in several works^5,32–34. Besides, they have been leveraged for post-architecture design tasks, such as generating NN parameters^35,36. Synthetic data generation for NN training has also thrived thanks to LLMs that can create diverse samples across different domains^6,37–39. While the main focus of these approaches is performance optimization, our work uses LLMs to generate a diverse dataset of NN architectures that can be leveraged by NN verification and migration methods.

Usage Notes

The dataset is intended to be used as a benchmark for evaluating tools related to neural network reliability and adaptability, including verification, refactoring, and code migration. Each Python file contains the source code of a single NN architecture defined as a PyTorch class, forming a complete and standalone implementation.

While the dataset covers a range of architecture types, tasks, input types, and complexity levels, several limitations should be acknowledged by users. Architectural complexity is defined in the dataset by depth and width of the networks, and topological patterns such as dense connections⁴⁰ and residual connections⁴¹ are outside the defined scope of this dataset. Multi-branch patterns are present in 40 out of 608 architectures. Additionally, the dataset is designed to cover classical architecture families including MLP, CNN, and RNN variants, and does not include Transformer-based architectures⁴², which are outside the defined scope of this version of the dataset. Furthermore, the dataset is generated following clean, structured PyTorch conventions, with no nested submodules or hierarchical module structures, making it a controlled baseline for the systematic evaluation of NN tools. Extending the dataset with real-world NN implementations in future work would allow for a more comprehensive evaluation across a broader range of coding patterns.

Author contributions

The authors of the manuscript have contributed equally to this work.

Funding

This project is supported by the Luxembourg National Research Fund (FNR) PEARL program, grant agreement 16544475.

Data availability

Our NN dataset is publicly available at Zenodo¹¹ under a CC BY 4.0 license.

Code availability

The code used to generate our dataset is publicly available in our repository (https://github.com/BESSER-PEARL/LLM-Generated-NN-Dataset).

Competing interests

The authors declare no competing interests.