In silico formulation optimization and particle engineering of pharmaceutical products using a generative artificial intelligence structure synthesis method

Abstract

Pharmaceutical drug dosage forms are critical for ensuring the effective and safe delivery of active pharmaceutical ingredients to patients. However, traditional formulation development often relies on extensive lab and animal experimentation, which can be time-consuming and costly. This manuscript presents a generative artificial intelligence method that creates digital versions of drug products from images of exemplar products. This approach employs an image generator guided by critical quality attributes, such as particle size and drug loading, to create realistic digital product variations that can be analyzed and optimized digitally. This paper shows how this method was validated through two case studies: one for the determination of the amount of material that will create a percolating network in an oral tablet product and another for the optimization of drug distribution in a long-acting HIV inhibitor implant. The results demonstrate that the generative AI method accurately predicts a percolation threshold of 4.2% weight of microcrystalline cellulose and generates implant formulations with controlled drug loading and particle size distributions. Comparisons with real samples reveal that the synthesized structures exhibit comparable particle size distributions and transport properties in release media.

Pharmaceutical drug dosage forms are traditionally determined through extensive physical experimentation. Here, the authors present a generative AI method that creates digital drug products from images, matching and improving critical quality attributes such as particle size and drug loading.

Introduction

A structure is an arrangement of matter. Typical pharmaceutical dosage forms have three design aspects: (1). Qualitative (referred to as Q1 hereafter), i.e., the choice of substances such as an active pharmaceutical ingredient (API) or a type of excipient; (2). Quantitative (referred to as Q2 hereafter), i.e., the amount of substance such as a loading of API; and (3). Structural (referred to as Q3 hereafter), i.e., the arrangement of the chosen substances in the chosen amount such as the particle size of the API and whether the API is spatially uniform throughout the dosage form. In pharmaceutical development, the interplay of Q1, Q2 and Q3 dictates the performance and quality of a drug product. For example, in the context of a polymer encapsulated drug product, the swell rate of a chosen polymer excipient (Q1), the amount of this polymer (Q2), and how the polymer is spatially arranged (Q3) will all impact the rate of API release from the drug product.

When designing a formulation to achieve desirable performance, iterative testing is often needed. Each iteration may include developing or modifying a manufacturing process, producing a product with a hope that its structures will meet the desired quality attributes, and determining the structure’s performance through experiments such as in vitro or in vivo dissolution testing. If the design process is based on trial and error, it can require an arbitrarily large number of iterations to achieve a structure with the correct properties to sustain therapeutic performance. For long-acting drug products where release profiles are measured in months, even a small number of iterations can be time and cost prohibitive. The later the development stage, the more costly each iteration can become. For example, a formulation change to support phase III clinical trials will potentially require costly bioequivalence studies and major updates to the manufacturing processes and analytical methods. Suboptimal formulation definition in early development can have a huge negative impact on later stage development. Conversely, formulation and dosage form choices are often considered non-critical in earlier development stages due to constraints in time and material availability. Pharmaceutical scientists are thus often stuck with large formulation uncertainties that proliferate the risk downstream due to the lack of cost and time effective tools. Therefore, minimizing the amount of physical manufacturing and experimentation can accelerate the development of the right structure in the early stages, and ultimately bring a product to the market in both a time and cost-efficient manner.

Structure imaging has already proven effective at reducing physical experimentation requirements, with in silico processing and simulations evaluating critical quality attributes (CQAs) in a fraction of the time and often with higher accuracy¹. However, collecting structure images from physical samples still requires time and expense on top of manufacturing of the samples to be imaged. Thus, the structure images of physical samples should be collected purposefully and reused thoroughly. Furthermore, a structure image generation method should be capable of generating structures with novel attributes while maintaining essential characteristics of the exemplar structures from existing images. For example, if the exemplar structure consists of a porous material compacted from crystalline particles, a generated structure with a higher porosity should consist of particles exhibiting intra-crystalline morphology and spatially distributed in a manner similar to the exemplar structure, rather than having a simplified geometry such as spheres dispersed randomly matching porosity and particle size.

While recent advances in artificial intelligence (AI) have impacted drug discovery and clinical data mining, tremendous opportunity also exists in drug development. This paper introduces a generative AI method that synthesizes formulations with structural features in silico from exemplar images of an existing, sub-optimal formulation. This method uses an image generator steered by input attributes typical to formulation development such as particle size and drug loading, to generate structural features of synthetic images with the specified input attributes. The generated attributes can either accurately match the exemplar attributes or vary according to user control, without being limited to attribute combinations that are represented in the exemplar data. This work demonstrates the potential of the generative AI method for extrapolation from existing formulations to new formulations that can be synthesized, analyzed, and optimized in silico. This in turn can potentially cut the costs for manufacturing or testing these new formulations, shorten their development cycle, and improve both environmental and social welfare.

This method² is an image-informed, AI-powered structure engineering method applied to pharmaceutical development. AI-powered image recognition has seen uses in pharmaceutical development, e.g., for material phase classification from various microscopy images, with one such workflow described later in this paper, or for quantification such as the Tox-RCNN³ method for measuring toxicity responses of cells from images. Our generative AI method was developed based on the foundation of generative AI for texture synthesis for non-pharmaceutical applications, which is briefly summarized below.

A texture is a function that defines a spatial arrangement of values, e.g., color, intensity, or label. Textures may exhibit a type of heterogeneity when comparing regions of limited size but converge to homogeneity or another type of heterogeneity as the field of view increases. This behavior is often referred to as stationarity and locality. A texture image is a discrete sampling of a texture over a bounded region.

Texture synthesis, specifically from exemplar image(s), is the numerical process of modeling an underlying texture that could have produced the exemplar texture image(s). Note that synthesis may also refer to subsequently sampling that solved texture to produce additional texture images.

Conditional texture synthesis similarly learns conditional textures, which take additional parameters that modulate attributes of the texture. For example, $f\left(\overrightarrow{x},t\right)=t<\sin x_1$ is a conditional texture, where the condition parameter $t$ controls, the volume fraction of the corresponding texture, e.g., $f\left(\overrightarrow{x},0\right)$ is a binary texture where half of the values in space are assigned phase 1, resulting in a volume fraction of 50%. To introduce controlled heterogeneity, condition parameters may be related to the spatial coordinates, e.g., $f\left(\overrightarrow{x},\mid \mid \overrightarrow{x}\mid \mid \right)$ causes the volume fraction to decrease as distance from the origin increases.

The objective of this research, in silico structure engineering for drug formulation development, is analogous conceptually to conditional texture synthesis by treating microstructure spatial distribution as a texture. Pre-existing methods for conditional texture synthesis were not readily applicable to structure engineering, due to limitations in control, speed, precision, accuracy, interpretability, quality, the amount and type of exemplar images that must be collected, and their ability to generalize attributes that are outside the range of attributes in the exemplar images.

For example, various machine learning and probabilistic algorithms, including Pyramid Histogram Matching⁴, Tree-Structured Vector Quantization⁵, and Gram Matrix Style Transfer with Neural Networks⁶, attempted to automate pattern detection and reproduction with stochastic variation, but often at a great computational cost, frequent failure due to patterns that cannot be represented in their hypothesis space, and little or no ability to directly and precisely control the attributes of the results. Tile-stitching approaches like Wang Tiles⁷ improved speed and reliability but lacked output diversity and offered still less control.

The deep-learning field has produced methods of generating arbitrary images with excellent diversity and control using gradient optimization of neural networks on a plurality of exemplar images. The most common methods include Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and Denoising Diffusion. For this research, GANs provided a satisfactory balance between method complexity and capability.

The original GAN⁸ architecture has numerous extensions to generate images with specific attributes, ranging from the earliest conditional GAN⁹ that sampled images from discrete classes (e.g., “dog,” “cat,” “plane”) to later methods such as StyleGAN¹⁰ that smoothly interpolated between classes by extracting “style” vectors from existing images, which encoded attributes of the image. However, these style vectors were a “black box” with no meaning to a human user nor a scientific/engineering application. An additional transformation function must be learned to map the style vector to meaningful controls – a task that is computationally expensive and algorithmically convoluted.

The Continuous-Conditional (cc)GAN¹¹ method addresses that issue by using interpretable scalar controls instead of reverse-engineering meaning from style vectors. The MO-PaDGAN¹² and PcDGAN¹³ methods expand on the ccGAN method to improve the accuracy and diversity of generated samples in multi-objective inverse design problems.

The ccGAN method specifies how to train a model, but not the model architecture. A suitable model architecture for structure engineering by texture synthesis should be capable of: (1) generating 3D structures (2) with representative spatial extents and (3) precise, interpretable conditional control, without requiring (4) excessive computation time or computer memory or (5) massive amounts of exemplar images. The last point is particularly important for new drug development as the structural morphology might only exist in one or very few exemplar, suboptimal product samples.

While our earlier efforts at applying texture synthesis to structure engineering employed the model from Non-Stationary Texture Synthesis by Adversarial Expansion¹⁴, and alternatives such as the Periodic Spatial GAN¹⁵, we found that the On-Demand Solid Texture Synthesis (STS)¹⁶ architecture was the best by far at satisfying the aforementioned requirements.

Solid Texture Synthesis refers to solving for 3D image volume textures from a set of 2D exemplar texture images. This is accomplished by the following inductive bias: a 3D image volume texture is a solution to a set of 2D exemplar texture images if and only if any set of 2D synthetic texture images sampled from the 3D synthetic image volume texture along any axis is similar to the set of 2D exemplar texture images.

On-Demand refers to this architecture’s ability to behave like a texture, specifically that any region of any size can be deterministically sampled at any time. The architecture can therefore be trained exclusively in 2D, and generate large contiguous texture images from smaller, independently computed chunks. These features alleviate concerns about resource requirements.

Neither On-Demand STS nor a subsequent GAN-trained STS¹⁷ included conditional control, which was achieved in this study by inserting Feature-wise Linear Modulation (FiLM) layers between the existing layers of the architecture.

The method employed in this paper, namely the novel combination of the ccGAN training method with an On-Demand STS model architecture augmented with FiLM layers, supports the functionality and performance requirements of pharmaceutical formulation development. We demonstrate the method’s verification and its applications in (a). determination of percolation threshold for an oral solid dosage formulation and (b). particle engineering for a long-acting implant. The in silico framework combining imaging, generative AI, and predictive modeling has potential for structure engineering in both pharmaceutical and non-pharmaceutical applications.

Results

Product sample digital transformation via imaging and image processing AI

The oral tablet and implant samples were digitized via a generalized workflow as shown in Fig. 1. The oral tablet specifically followed the workflow in Fig. 1a1–1g1. The implant followed the workflow in Fig. 1a2–1g2. Dosage agnostic workflow diagram is illustrated in Fig. 1h.

Fig. 1. Digital transformation workflow h and i, exemplified by two case studies row a1-g1 and row a2-g2.

Each step in row h is aligned to the corresponding steps in the top two rows. a1 Photo of an oral tablet sample in a container. b1 Illustration of XRM imaging principle employed to image the oral tablet sample. c1 One cross section of XRM image volume of the oral tablet sample. d1 Visualization of a digital twin of the oral tablet sample reconstructed from the 3D XRM image volume. e1 3D MCC network extracted from the digital twin Figure 1d1 via AI image processing. f1 Throat size distribution of MCC network computed by in silico mercury intrusion capillary porosimetry. g1 Water uptake computed by in silico mercury intrusion capillary porosimetry. a2 Photo of a polymer implant sample. b2 Illustration of FIB-SEM imaging principle. c2 One cross section of the 3D FIB-SEM image volume. d2 Visualization of a digital twin of the implant sample reconstructed from the 3D FIB-SEM image volume. e2 3D API network extraction from the digital twin via AI image processing. f2 API particle size distribution computed from (e2). g2 Predicted in silico release profile. h Digital transformation workflow. i Synthetic formulations created by the generative AI method in this paper. XRM X-ray microscopy, FIB-SEM focused ion beam scanning electron microscopy, MCC microcrystalline cellulose, API active pharmaceutical ingredients, AI artificial intelligence, PSD particle size distribution.

Oral product

Four oral tablet samples were digitally transformed by X-Ray Microscopy (XRM) imaging to train and validate the structure synthesis generative AI. These tablets consist of two main components: dicalcium phosphate dihydrate (DCP, Di-Tab, Innophos Inc., IL, USA) and microcrystalline cellulose (MCC, Avicel PH 101, FMC Biopolymer, PA, USA). All tablets contain 0.5% magnesium stearate (MgSt, Ligamed MF-2-V; Peter Greven GmbH, Venlo, Netherlands) for internal lubrication. Formulation compositions of the tablets are provided in Table 1.

Table 1.

Formulation compositions of tablets under study

Tablets	MCC %wt	DCP %wt	MgSt %wt	Visual
TB05	5	94.5	0.5	Fig. 2a
TB10	10	89.5	0.5	Fig. 2b
TB20	20	79.5	0.5	Fig. 2c
TB30	30	69.5	0.5	Fig. 2d

TB Tablet, MCC microcrystalline cellulose, DCP dicalcium phosphate dihydrate, MgSt magnesium stearate, %wt percent by weight.

The percolation threshold is an important attribute for material microstructure development. How much material will form a connected network depends on the primary material properties (e.g., API particle size and shape), the secondary materials properties (e.g., excipient particle size and shape), the formulation (e.g., amount of API), and the process (e.g., compaction, granulation, or hot-melt extrusion). Conventional approaches use trial and error, wherein achieving the desired accuracy involves substantial resources in product manufacturing and characterization. In this particular study, the structure synthesis generative AI is used to determine the amount of MCC that will form a percolation network, which is extremely difficult experimentally.

Figure 2 shows one cross section of the 3D XRM image volumes from each of the 4 oral tablet samples. The increasing amount of MCC, shown as the darker gray phase, is apparent. The training and verification were conducted using tablet samples TB10, TB20, and TB30, following the workflow illustrated in Fig. 3.

Fig. 2. Central top-down cross section view of XRM image volumes acquired on four DCP/MCC tablet formulations shown in Table 1.

Light gray corresponds to higher density DCP material domain. Dark gray corresponds to lower density MCC domain. a TB05: Tablet with 5%wt MCC. b TB10: Tablet with 10%wt MCC. c TB20 Tablet with 20%wt MCC. d TB30 Tablet with 30%wt MCC. XRM X-ray microscopy, MCC microcrystalline cellulose, DCP dicalcium phosphate dihydrate, TB tablet, %wt percent by weight.

Fig. 3. Oral formulation structure synthesis training and verification workflow.

a A cross section of the segmented image volume of TB10%, overlaid on top of the same cross section of the greyscale XRM image volume. b Same as (a) for TB30%. c–f A cross section of the numerically synthesized image volume, with increasing %wt of 9.6 (c), 12.8 (d), 15.3 (e) and 20.1 (f). g Same as (a) for TB20%, not exposed to the structure synthesis generative AI. h The cross section of segmentation (g) in binary display. 3D image volumes of (f) and (h) are then subjected to CQA and transport property comparisons to assess the physical similarity of the AI generated formulation (f) to the corresponding real sample (h) that was not seen by the AI during the training. XRM X-ray microscopy, MCC microcrystalline cellulose, DCP dicalcium phosphate dihydrate, CQA critical quality attributes, AI artificial intelligence, %wt percent by weight.

The images from all the samples were subjected to an identical AI image processing workflow. The 3D segmentations of TB10 and TB30 tablets, as shown in Fig. 3a, b respectively, were input to the structure synthesis generative AI as training data. After training, the structure synthesis AI was used to generate a sequence of in silico 3D image volumes of tablets with varying amounts of MCC, as shown by a few example cross sections in Fig. 3c–f. Each in silico tablet image has a volume of 1000x1000x1000 voxels, similar to the XRM imaging data on TB10 and TB30. One such compact with 20.1%wt MCC (Fig. 3f) was selected and designated as the “AI” tablet for verification.

Independently, TB20, after segmentation (Fig. 3g), was binarized (Fig. 3h) and designated as the “real” tablet for verification, since this data was not used to train the structure synthesis AI.

The MCC phase of the AI tablet (Fig. 3f) and real tablet (Fig. 3h) were then subjected to three quality attribute characterizations: particle size distribution (PSD), in silico MICP simulation, and in silico permeability simulation. The comparisons are shown in Fig. 4. It should be noted that all calculations in Fig. 4 used identical parameters, thus the difference can only come from the microstructure properties between the real and AI datasets.

Fig. 4. Oral formulation structure synthesis verification results by comparing the real MCC network in Fig. 3h to the AI network in Fig. 3f.

a Particle size distribution of the MCC phase. b In silico mercury intrusion capillary porosimetry of the MCC phase. c In silico permeability of the MCC phase. Source data are provided as a Source Data file. DCP Dicalcium phosphate dihydrate, MCC microcrystalline cellulose, AI artificial intelligence.

Figure 4a compared the MCC material PSD calculations of 3D image volumes from XRM images of the real 20%wt sample and AI generated sample. For each synthetic tablet, the PSD calculation was computed using a 2D slice by slice method (i.e., treating the 3D image volume as a stack of 2D images, performing the calculations on each 2D image, before combining many 2D calculations into one single PSD using equivalent circular diameter), and a 3D volume method (i.e., treating the 3D image volume as an image volume, performing the calculations in 3D). The 2D slice by slice calculations are nearly identical between real and AI image volumes in D₁₀, D₅₀, and D₉₀, except that the AI (2D) PSD is less than 5% smaller than the real (2D) PSD toward the larger particle size D₉₀. The 3D image volume PSD calculations show a consistent trend as in 2D, where the AI (3D) PSD is generally comparable with real (3D) PSD. AI (3D) PSD is smaller than the real (3D) PSD, and the difference is also more pronounced in the larger particle range, i.e., there is no difference in D₁₀, ~5% reduction in D₅₀, and ~20% reduction in D₉₀. This is not uncommon as D₉₀ is inherently more variable than D₅₀ and D₁₀ due to particle aggregation.

Since the structure synthesis AI is trained using 2D images, it is expected that PSDs calculated using 2D method compare more favorably than PSDs calculated using 3D method. The PSD, however, only characterizes the domain size of a material phase. It does not tell the full story of the network of a phase. Fig. 4b, c characterize how the MCC network is connected. In Fig. 4b, simulated mercury fluid was forced into the MCC network (X- direction) by increasing capillary pressure (Y+ direction). This numerical simulation assumed the MCC network contained no material, thus the capillary pressure response only depends on the topology, connectivity, and tortuosity of the network. The two Pc curves from the real and AI tablets are nearly identical, except at the beginning and end of the mercury intrusion. The beginning of the intrusion shows the real tablet had slightly lower intrusion pressure, indicating slightly larger MCC network throats near the surface of the tablet. Toward the end of the intrusion, the network throats in the real tablet were smaller than the AI tablet, i.e., they required higher intrusion pressure. Fig. 4c shows permeabilities calculated from both the real and AI tablets using computational fluid dynamics flow simulation through the MCC network. For these calculations it is assumed that flow resistance only come from the MCC network topology, connectivity, and tortuosity (similar to what was assumed for MICP simulation). A vector of three permeability scalars along 3 spatial orientations, $k_x,k_y,k_z$ were calculated, then converted into a scalar permeability magnitude $k_n=\sqrt[2]{k_x^2+k_y^2{+k}_z^2}$. The four scalar permeabilities are comparable between the real and AI tablets, with the AI tablet showing slightly lower permeability values than the real tablet.

Figure 5c shows the structure synthesis AI predicted 4.2%wt MCC as the exact percolation threshold. In other words, the AI predicts that an MCC amount below 4.2%wt will not form a connected, i.e., percolating, network. Above 4.2%wt, percolation is established with a high tortuosity. As the MCC amount increases, tortuosity decreases exponentially. The tortuosity arising from the four real tablets are plotted as orange squares for further verification. Most notably, a tablet with 2%wt MCC was made after the generative AI training (Fig. 5a). Tortuosity calculations from its corresponding segmentation (Fig. 5b), using the segmentation modules identical to the ones used for 5%wt, 10%wt, 20%wt and 30%wt tablets, returned no tortuosity. 5%wt (>4.2%wt) with a tortuosity and 2%wt (<4.2%wt) with no tortuosity validated the percolation threshold predicted by the generative AI.

Fig. 5. Percolation threshold prediction and validation based on AI-generated synthetic structures.

a 2%wt tablet made for validation post generative AI training. One cross section of the 3D XRM image volume of the tablet sample was shown. b The same image as (a), overlaid with the segmentation of MCC phase (green) and DCP phase (blue). c Tortuosity calculations of all real tablet volumes (green squares) and AI synthesized volumes (red circles). 4.2%wt MCC was predicted as the exact percolation threshold, shown by the right boundary of the region shaded in green. Source data are provided as a Source Data file. XRM X-ray microscopy, AI artificial intelligence, MCC microcrystalline cellulose, DCP dicalcium phosphate dihydrate, %wt percent by weight.

Implant product

In a long-acting formulation targeting HIV PrEP, crystalline drug Islatravir, a nucleoside reverse transcriptase translocation inhibitor (NRTTI), was dispersed into a poly(ethylene-vinyl acetate) (EVA) polymer matrix to form an implant through a hot melt extrusion (HME) process¹⁸. The API distribution heterogeneity within the implant was observed to have a strong dependency on the API synthesis chemistry, particle size reduction approaches, and extrusion process parameters. The structure synthesis generative AI was applied to understand the impact of API distribution heterogeneity to release and de-risk particle engineering decisions.

Figure 6 shows 9 implant samples numerically synthesized, arranged in a 3 × 3 array. The rows of the array indicate the radial distribution of drug loading, while the columns of the array indicate the radial distribution of particle size. Using one focused ion beam scanning electron microscopy (FIB-SEM) image from an implant sample, a uniform in silico implant was numerically synthesized with the structure synthesis AI (Fig. 6e in the center, API particles are white). Keeping the drug loading and PSD in the whole implant system identical, 8 numerically synthesized implants were generated by varying drug loading and PSD radially. From sample to sample along the horizontal axis, the drug loading was increasing (Fig. 6d) or decreasing (Fig. 6f) radially outward. On the vertical axis, the API particle size was increasing (Fig. 6b) or decreasing (Fig. 6h) radially outward. Fig. 6a has both more drug loading and larger particle size in the outer portion of the implant than the core portion. Fig. 6i symmetrically has both less drug loading and smaller particle size in outer portion of the implant than the core portion. Fig. 6c has larger particle size but less drug loading in the outer portion of the implant, while Fig. 6g has smaller particle size but more drug loading in the outer portion of the implant.

Fig. 6. Structure synthesis results for implant formulation design and particle engineering.

All 9 implants were digitally synthesized with overall API PSD and drug loading similar to the real implant used in training. a A cross section of an AI synthesized 3D implant where larger particles and higher drug loading are distributed away from the inner core towards the exterior boundary. b Same as (a), except that drug loading is uniformly distributed. c Same as (a), except that higher drug loading is distributed towards the inner core. d Same as (a), except that particles of all sizes are uniformly distributed. e Same as (d), except that drug loading is uniformly distributed. f Same as (d), except that higher drug loading is distributed towards the inner core. g Same as (a), except that larger particles are distributed towards the inner core. h Same as (g), except that drug loading is uniformly distributed. i Same as (g), except that higher drug loading is distributed towards the inner core. AI artificial intelligence, API active pharmaceutical ingredient, PSD particle size distribution.

Combined with a validated in silico release prediction method^1,18, the synthetic implant formulations in Fig. 6 generated in silico scenarios to evaluate the process impact and risks to performance. Figure 7 shows the in silico release profile predictions on 3 of the 9 implants with a designed release duration of 24 months. Formulation Fig. 6b has larger particles near the outside of the implant, thus showing a slower burst release at the same drug load. For this biostable EVA implant system, the diffusion of the API through porous network from dissolved API is the only drug transport pathway. Thus larger API produces a less efficient percolation network. Formulation Fig. 6h has smaller particles near the outside, thus showing a faster burst release. The formulation with uniform dispersion, Fig. 6e, shows a burst release similar to formulation Fig. 6h, then transitions into a late-stage release similar to formulation Fig. 6b.

Fig. 7. Release profile variations associated with spatial variations of API particles.

In Silico release profile simulations on 3 selected implant formulations corresponding to 3 of the digital implants shown in Fig. 6. Source data are provided as a Source Data file.

Discussion

Advantages

The structure synthesis generative AI reported in this paper has a few immediate advantages to imaging. First, it can generate an arbitrarily large field of view (FOV) quickly and inexpensively, whereas the FOV from physical imaging experiment is limited by detector size and resolution, and mosaic microscopy approaches to increase the imaging FOV are costly. Second, it enables generation of a 3D image volume, also with large FOV, from one or more 2D examplar images. Beyond dimensional expansion, this 2D image to 3D image volume synthesis solves artifacts that are intrinsic to the planar 2D imaging approaches used to reconstruct 3D volumes. In 3D FIB-SEM and other cut and image approaches specifically, the image volume often exhibits a combination of anisotropic depth resolution, alignment artifacts, “pore back” artifacts due to signals traveling behind imaging plane, and milling artifacts. With 2D to 3D synthesis, a subset of ‘clean’ slices can be used to train a generator that produces an artifact-free volume with isotropic resolution.

Conditional control is a third advantage with substantial potential in formulation and process development, discussed in the next section. Instead of mimicking the exact microstructure of an existing image, certain properties (e.g., drug loading) of the microstructure morphology can be modified, while certain other properties (e.g., drug particle size) are kept constant.

Formulation design, process development, and regulatory implications

The percolation threshold determination demonstrated in Fig. 5 has broad applications in formulation development. A majority of dissolution and release problems are associated with percolation. Controlled drug release systems using porous diffusion rate control require a percolation network of API, pore former, or polymer matrix respectively to function. Such controlled release systems include long-acting microspheres¹⁹, non-degradable or weakly biodegradable implants²⁰, and intra-uterine systems²¹, oral controlled release membranes or matrices with pore formers²², or intra-polymer matrix diffusion. Specifically, drug loading has been a formulation design parameter. The generative AI method found successful application to optimize the amount of drug that can sustain both a sufficiently high therapeutic window and dose safety to patients. Erosion controlled release does not directly depend on percolation, though the API dispersion pattern still influences release rates and can be studied with the structure synthesis method. Immediate release formulations might rely on a percolating disintegrant network²³, or a percolating porosity network, to achieve desirable disintegration performance. Whether the amorphous compound rendered in the polymer matrix forms a percolation network will dictate both its dissolution performance and stability. In fact, Leuenberger²⁴ suggested that the principle of percolation has a broad applicability to multi-component systems. Many properties (X) of such systems can be described by a power law expression of the percolation theory as: $X=S{\left(p-p_c\right)}^q$, where S is a scaling factor, p is the occupation probability, $p_c$ is the percolation threshold, and q is the exponent. Leuenberger argued that a robust formulation is such that the relative composition p is not too close to $p_c$. Despite the soundness of this ideal, the principle of percolation was not extensively employed in formulation development, partly due to the complex microstructure of dosage forms and the challenges to accurately identify percolation thresholds. The generative AI methodology has thus opened an opportunity to incorporate the percolation theory in future drug product development practices.

In addition to percolation, other formulation development questions may be addressed by the generative AI tool. For example, variation of porosity and compositions of material phases such as API or functional excipients can corroborate the dissolution, stability, and tensile strength of a formulated product. Additionally, particle size can be reverse engineered in silico from a desirable performance matrix.

The process-introduced heterogeneity demonstrated in Fig. 6 can help answer “what if” questions in process development and optimization. Particle distribution in the drug product can be heterogeneous from multiple sources. For example, during hot-melt extrusion, a laminar flow profile can preferentially distribute larger particles toward the outer surface of an extruded rod, while keeping smaller particles in the core. During tablet compaction, it was known that a heterogeneous density distribution is prevalent in tablets possessing a non-flat surface²⁵. Compactor tooling designs with a leading edge (touching the power bed earlier), a trailing edge (compacting the powder bed later), and imprint logo letters can also cause heterogeneity. Tools to conduct such “what if” investigations can help purposefully achieve a desired performance, such as drug release or mechanical strength, by design, eliminate problematic processing conditions earlier on, and reverse engineer failure to provide solutions efficiently.

This structure synthesis generative AI tool addresses long standing challenges in the FDA’s microstructure (Q3) sameness assessment. While qualitative (Q1) and quantitative (Q2) sameness are relatively straightforward to achieve and confirm, Q3 sameness, i.e., the sameness of distribution of materials in two drug products, either brand products in different clinical or marketing phases, or generic products in comparison with brand products, demands non-destructive, highly discriminatory, and 3D characterization tools such as XRM. These imaging tools answer whether two products are the same or not. This structure synthesis tool will advance one step further by answering why they are different, and how they can be made the same.

Limitations and future work

Due to the relative novelty of this tool, validation (i.e., comparison with experimental data) and verification (i.e., comparison within silico data using other methods) both require significant additional effort. The authors specifically emphasize that method validation in pharmaceutical development, often under regulatory filing context, is non-trivial and may require good manufacturing, laboratory, and data practice (GxP) protocols. Initial validation efforts for both the oral tablet and the implants are expanded into near future publications where complex validation protocols involve potentially in vitro, in vivo, and in silico elements. Transitioning this tool from research and development to production and filing requires time and close collaboration among pharmaceutical scientists, process engineers, computational scientists, and regulatory scientists.

The structure synthesis generative AI workflow is limited to synthesizing material structures with explicit phase definition. Such definition needs to be captured in a image and segmented afterwards. Although the images used in this paper were limited to XRM and FIB-SEM, any imaging modality that can capture distinct material phases is supported. The segmented image is currently limited to binary phase, though multiple phases with conditional controls of each phase will be supported in the near future. However, it is worth noting that for practical purposes, multi-component systems may be reduced to binary systems for a particular function of interest, for example solids (combining all ingrediences into one phase) and pores (combining fractures, voids, and intraparticle micropores), or soluble components and insoluble components. Future work includes structure synthesis from images that were hand drawn, or images generated from another in silico method when no imaging method can capture the structural arrangement physically. One example is in amorphous solid dispersion (ASD). How amorphous molecules are dispersed in the hosting polymer matrix at the molecular level is very difficult, if not impossible, to image directly. Such distribution, expected to impact the bioavailability and stability of an ASD system, can be conceptually depicted via in silico images. Structural synthesis tools can then generate the evolution of depicted structures with varying process and formulation conditions or under varying stability conditions.

The structure synthesis generative AI tool is trained on existing images. While it has the advantage of learning from real-world, physical information, it inherits the limitations intrinsic to imaging. Structures that are not captured by imaging cannot be reproduced by the structure synthesis AI without, for example, a correlative, multi-resolution approach. Furthermore, the training image(s) must contain a representative amount of the desired structures and features with sufficient variation to ensure effective training. For the binary systems illustrated in the preceding examples, this notion translates to the need to encompass a large enough number of particles and a field of view containing a representative particle dispersion pattern. As a general guideline, all relevant imaging data that is available should be used to ensure the best results possible, though structure synthesis has been successfully used with training data typically equivalent to a 1000-Cube voxel volume, and even as little as a 300-Cube voxel volume.

The two generative AIs for numerical structure synthesis, one for oral tablet and one for implant, were trained with limited data. More training will greatly enhance robustness and predictive accuracy. On the other hand, this generative AI is designed to function with limited known samples. The cost and resource constraints in the industry often leave scientists and business leaders with limited information to make decisions. For example, when only a small amount (e.g., a few mg) of API was available during the early stage of a typical discovery program, physically making 6 samples is both financially and logistically challenging. Generative AI trained on 1 or 2 microsphere samples can synthesize numerically microsphere, implant, depot, and various dosage forms without incurring material nor physical testing cost, thus can go a long way to support formulation and process decisions via feasibility assessments, rank orderings, and risk predictions. Training set expansion is further complicated by data confidentiality. In certain areas, such as generic drug development, generative AIs publicly funded by the FDA can support generic product development publicly. In certain other areas, such as new drug development, generative AIs has to carefully compartmentalize according to each individual company’s boundaries of IP and confidentiality. To what extent can that training dataset be considered minimally sufficient, and further, what is defined as sufficient, are both important topics requiring further research and validation. It is the authors’ view that a practically successful and responsible generative AI needs to function and create value based on limited training data.

For the two products, a generative AI model is needed for each due to their largely different material properties and manufacturing process. The extrapolation of a generative AI model trained on one product to another is an interesting and exciting application, which requires further research and validation.

The API particle size distribution within the implant sample depends on the API properties, particle size reduction approaches, and extrusion process parameters. Glass transition temperature of API can play a significant role in HME. While these additional investigations will be reported in future publications, the synthetic implant samples generated assumed that API particle change (or no change) is the same. If API particles change chemically, both the training data selection and the training strategy need to be revised.

In silico optimized microstructures using generative AI may not be always achievable in reality. For example, the structure that AI generated can exist physically, however, there might not be a viable way to achieve such structure via existing methods of API chemical synthesis, particle processing, and manufacturing. Decision makers need to be made aware of the risks that generative AI can propose an optimized structure that either cannot be practically realized, or require substantial investment in modifying existing engineering practices. Conversely, the availability of such generative AI tools can drive innovations in the aforementioned areas.

While generative AI methods pose great potential to probe structure-performance relationships for pharmaceutical dosage forms, its capability probing structure-performance relationships over time, possibly under different dissolution environments, requires additional effort. Imaging, with careful design of experiments, can capture dissolution performance and more importantly, the biopharmaceutics performance, which can be further quantified, mechanistically investigated, and predictively modeled²⁶. Imaging data of temporal evolution of microstructures, subjected to dissolution, stability stress conditions with temperature and humidity, and mechanical force, can be part of the training data to the proposed generative AI workflow. It opens up opportunities to probe spatiotemporal relationships and explore the valid range of predictive capacity on the impact of changing spatial distribution of components have on final product performance compared to the structure at t = 0.

In addition to training using temporal correlation imaging data, i.e., data acquired over time under aforementioned conditions, the training can also be conducted using process correlation (e.g, changes during tablet compaction elucidating particle morphology and interlocking mechanism evoluation²⁷), scale correlation (e.g, intra-particle characterization via FIB-SEM and inter-particle characterization via XRM²⁸), modality correlation (e.g, X-ray and Raman), and structural-chemical correlation (e.g., particle morphology via SEM and particle chemical composition via energy dispersion (X-Ray) spectroscopy (EDX)²⁹.

Closing remarks

A generative AI method that can design drug formulations with structural features from XRM and FIB-SEM images of exemplar formulations was introduced. This generative AI method was (a). validated with an oral dosage formulation and used to find the percolation threshold of MCC network and (b) employed to study in silico a long-acting implant product where process introduces radial heterogeneity. Synthetic structures with predicted properties can be generated by this method to assess “what if” questions, and to drive rapid and cost-effective process and product optimization. With further validations and improvements, it is envisaged that the structure synthesis generative AI tool will help address long standing challenges including percolation prediction, particle engineering, formulation and process optimization, and the microstructure (Q3) sameness assessment.

Methods

Oral tablet formulation

Blends shown in Table 1 were compressed into tablets using a bench-top compaction analyzer (Gamlen D1000, Gamlen Tableting Ltd, London, UK) at the target solid fraction of 0.85. The upper punch movement followed a linear profile at 1 mm/s, while the lower punch remained stationary. A 6-mm round, flat-face tooling was used. The weight of the tablets were adjusted such that all tablets exhibited the same thickness (2.75 mm) at the target solid fraction.

Implant formulation

The crystalline Islatravir API and EVA polymer were selected for formulation development owing to their suitable physical and chemical properties. The polymer and API particles were blended at 40%wt. The pre-blends were extruded at a barrel temperature of 140 °C. Finally, the implant sample is digitally transformed via FIB-SEM and used to train and validate the generative AI that synthesize implant structures numerically.

Tablet XRM imaging

XRM was determined as the most appropriate imaging technique, as it can capture the entire tablet non-invasively, while achieving sufficient resolution to resolve the MCC particle network²⁷. Oral tablet samples were mounted on a rotational stage inside a Zeiss Versa 520 XRM system. A region of interest (ROI) was selected to cover the entire tablet, and one x-ray radiograph was taken. The sample was then rotated 1600 times each followed by one additional x-ray radiograph, until it completed a 360-degree rotation. The integration time used was 0.5 s with an X-ray source energy of 80 keV. A filtered backward projection algorithm was used to reconstruct the radiographs into a stack of approximately 1000 images, with each image measuring 1000 by 1000 voxels and a voxel size of 2 µm.

All images collected are managed by digiM I2S enterprise cloud software platform, v4.5. Subsequential iterative AI image segmentation, image analysis algorithm execution, predictive modeling, AI training, computing resource support, and data hosting, are conducted under digiM I2S ³⁰.

Implant FIB-SEM imaging

FIB-SEM was determined as the appropriate imaging technique as the primary particle size (in the order of 1–3 um) requires high resolution to resolve¹⁸. The implant sample was coated with platinum and imaged via a Zeiss Auriga cross beam FIB-SEM system¹⁸. Eight hundred slices were collected for a sample volume of approximately 60 × 40 × 24 µm³, using a 30 keV Ga+ focused ion beam for thin section milling at 30 nm thickness, and a 2 keV electron beam for mixed backscattered and secondary electron imaging at 30 nm spatial resolution.

Iterative AI segmentation

The XRM and FIB-SEM images were segmented using an iterative supervised machine learning (SML) and deep learning approach. An experienced user selects a small crop of one or more 2D images which contain all desired features as training images and draws one or more trace lines over the pixels that belong to each feature. A random forest classifier then classifies each pixel of the training image(s). The user iteratively adds training traces until a reasonable segmentation has been achieved. The user then applies the classifier to additional images. If needed, selected SML segmented images can be modified to train a convolution neural network deep learning model, which can be iteratively applied to more images. Depending on the complexity of the images, one to five iterations are often needed to achieve satisfactory segmentations. Tablet XRM 3D image volumes were segmented using one SML classifier, with exemplar cross section segmentations overlaid on the corresponding greyscale images shown in Fig. 3a, b, g. Implant FIB-SEM imaging required 5 steps: an initial SML segmentation on 1 cross section image, a manual correction, a deep learning training on the 1 corrected image followed by segmentation on 3 different cross section images, a manual correction, and a final deep learning training on 4 corrected images followed by segmentation on the entire stack of 800 images. Example segmentations can be found in ref. ¹⁸.

Quality and performance attributes

Volume percentages of a particular material are computed by dividing the number of voxels corresponding to the material by the number of voxels corresponding to the sample.

Particle size distribution (PSD) is calculated by the identification of individual particles. When the particles are isolated, individual particles can be easily identified via the connected voxels. However, when the MCC particles connect into each other and form a continuous network, particle size distribution is not readily defined. Such connections are either physical when the volume % reaches or exceeds percolation threshold, or numerical due to limited resolution. In the latter case, when the spacing between two separated particles is smaller than the voxel size of the imaging volume, the two particles will appear to be connected under that resolution. To compute a particle size distribution from connected network, a watershed algorithm³¹ is needed to split the network at the narrowed connections between two adjacent particles. Finally, a statistical binning of equivalent spherical diameter of all the identified individual particles is performed to generate PSD.

In Silico mercury intrusion capillary porosimetry (MICP) was simulated using a morphological method of smallest inscribing spheres that can move through a connected network. The measured sphere size is correlated with capillary pressure by Youngs Laplace Equation³².

In Silico permeability was simulated using a voxel-based Navier-Stokes solver using a finite volume discretization, where the flux was computed from momentum equations before integration into a permeability tensor using Darcy’s law^33,34,

In Silico release profile was computed using a combination of percolation simulation, effective diffusivity simulation via computational fluid dynamics method similar to permeability but solving Fick’s second law, and a time conversion using Higuchi’s equations^1,18.

Structure synthesis with generative AI

The generative AI method to engineer new microstructures from existing images using conditional texture synthesis at a pragmatic application level² is summarized in Fig. 8. The method consists of training an enhanced On-Demand Solid Texture Synthesis generator model with the ccGAN method.

Fig. 8. A schematic workflow diagram of the generative AI method for structure synthesis, which is trained from exemplar images and the CQAs extracted in the training workflow, then applied to generate synthetic formulations in the prediction workflow.

a Microstructure images from a real sample. b Ground truth values of CQAs to be controlled for each ground truth image (a). c Generator AI that will generate a synthetic image. d Compare AI that compares the ground truth images (a) with the synthesized images (l), and produces an assessment of (e) quantitative similarity. f Measure AI that estimates the same set of CQAs in (b) from generator output (l) and produces an assessment of (g) quantitative accuracy. h Iterative adversarial optimization. i The generative AI model after training. j User prediction input with specified CQA. k Final prediction output with qualitative similarity to training input images (a), and quantitative accuracy to user prediction input (j). CQA critical quality attribute; AI artificial intelligence; %wt percent by weight.

Training dataset and architectural considerations

The method requires one or more segmented structure images as training data. If the structure images contain some irregularly shaped solid object, such as an oral solid dosage tablet scanned by the cylindrical field of view from an XRM scan, the background that is not part of the tablet interior needs to be masked out. This ensures that features such as the object boundary are not randomly permuted throughout the learned texture. If the irregular shape is critical for analysis, it can be reintroduced later.

Next, a set of uniformly sized 2D square tiles (Fig. 8a) are randomly sampled from the structure image(s). Since only 2D tiles are required, the structure images can also be 2D, e.g., anisotropic FIB-SEM stacks or MFV images. In this work, only isotropic 3D image volumes were used, with tiles sampled from all three directions. The size of the tiles is selected to ensure both a representative count of morphological features in every tile, and a non-representative homogeneity within any spatial heterogeneity. The size is further constrained by GPU and computing resources. Each tile is then processed with standard analysis algorithms to determine the values of its local attributes, e.g., volume fraction of a phase. This collection of tiles each paired with their corresponding attributes comprises the training dataset.

Training on tiles and the corresponding local attributes does have some significant side effects. The first is that limited-size tiles can exhibit large sampling variance, i.e., a structure image with nearly homogeneous porosity could yield tiles with arbitrary porosities if the tiles were sufficiently small, however excessively small tiles would fail to produce realistic morphology. In this project, we found sampling variance beneficial, as it smooths and expands the distribution of attributes available for the model to learn.

A second side effect is that the learned texture’s morphology can be under-constrained by the tiles. For example, a volume consisting of uniformly sized spheres would yield 2D cross section tiles containing randomly sized circles. Then, a possible 3D solution to that set of tiles is a volume consisting of randomly sized spheres, which is not the correct solution. This effect necessitates verifying the trained texture model against real data to measure the severity of morphological differences in terms of their effect on critical quality attributes (CQAs).

A third side effect is that the model only ‘understands’ the attributes in their 2D form. For some attributes such as porosity, 2D understanding is sufficient. For particle size distribution, the difference between 2D and 3D understandings can be significant. For example, querying the trained model for a volume where particles have an average size of 50 microns is querying for a volume where any slice’s cross-sections of particles have an average equivalent circular radius of 50 microns. It is possible to compensate for this kind of 2D-3D attribute discrepancy by repeatedly querying the model with different 2D image input attributes until the resulting volume, when analyzed, has the correct 3D image volume attributes. We automated this compensation with Bayesian Optimization, which searched for input attributes that minimized the distance between the model output’s 3D attributes and the target attributes.

Training workflow

The ccGAN training method employed in this work, like all GAN methods, consists of a training dataset of “real” images – such as the dataset constructed in the prior section – and two gradient-optimizable neural networks: a generator network (Fig. 8c) that produces numerically synthesized random images (denoted as “synthesized” hereafter for easier narration), and a discriminator network that estimates the likelihood that a given input image is real or synthesized, i.e., a learnable metric for qualitative similarity. The training process alternates between optimizing the generator to minimize the discrepancies that the discriminator uses to identify synthesized images, then optimizing the discriminator to detect remaining discrepancies between real and synthesized images. This process converges when the discriminator can no longer identify any discrepancies between real and synthesized images within an acceptable tolerance.

The use of “qualitative similarity” may imply a degree of subjectivity. Indeed, it would be subjective if human judgment were involved in evaluating it. However, as used in this work, qualitative simply refers to any aspects of the image(s) that are not explicitly measured or controlled. These unspecified aspects are still measured quantitatively by the trained discriminator, but only as a fuzzy combined value representing statistical equivalence to the real data. Many of these unspecified aspects could be measured or controlled quantitatively as CQAs, e.g., mean inter-particle distance, circularity, roughness, etc.

Generating synthesized images with the generator during training employs the following process. A set of real images is randomly selected from the constructed training dataset (Fig. 8a), and a small amount of random noise is added to each of those real samples’ attributes (Fig. 8b). The displaced set of attributes is the input to the generator (Fig. 8c) to produce corresponding synthesized images (Fig. 8l). The On-Demand STS generator architecture requires an additional input of random noise at multiple scales, which the model learns to transform into tiles with the correct textures for those attributes. The resulting synthesized tiles are each paired with the input attributes that produced them, thus comprising a batch of synthesized samples.

For successful training in this project, a basic qualitative similarity metric is not sufficient. It must be possible to evaluate quantitative similarity between a given 2D image tile and some attributes. Two common strategies for quantitative similarity metrics in GANs are conditional discriminators and auxiliary regression classifiers. In the implementation and testing of both, we’ve found auxiliary regression classifier helps to maximize extrapolation from a single image at the cost of some morphological realism. It is generally advisable to collect imaging data from a second or third sample rather than relying on a single sample and maximizing extrapolation, so the remainder of this section discusses primarily the conditional discriminator.

A conditional discriminator learns to measure quantitative similarity by measuring conditional qualitative similarity. Given a generated 2D image tile paired with some attribute values, it estimates the similarity of that 2D image tile to the real 2D image tiles labeled with similar attribute values. Since the real image tiles are all labeled with their real attributes, i.e., the attributes were directly computed from the real image tiles, when the discriminator receives as input a generated image tile consisting of, e.g., an image with 90% actual porosity, but labeled as having 10% porosity, it knows that sample is synthesized, and the generator must learn to correct that discrepancy. This unified model replaces the separate models (Fig. 8d, f) for comparing and measuring images by producing a single metric that combines qualitative (Fig. 8e) and quantitative (Fig. 8g) similarity.

When optimizing the generator, that batch of synthesized images is the input to the conditional discriminator, which yields a scalar score for each synthesized image representing the qualitative (Fig. 8d) and quantitative (Fig. 8f) similarity of the sample to the real images, where positive scores represent higher realism. Since the convergence objective is to be indistinguishable, the generator loss, i.e., error, function is simply the squared magnitude of the scalar scores, summed together across the batch. That loss is minimized using backpropagation and gradient descent to update the generator.

The conditional discriminator is further optimized iteratively (Fig. 8h). Both the batch of synthesized images and batch of real images are input to it to again produce scalar scores. However, the loss function for the synthetic images minimizes the difference between the scalar score reported by the discriminator and negative 1, while the loss function for the real samples minimizes the difference between the scalar score of real images and positive 1. The combined loss function for the conditional discriminator is the sum of the losses over the synthetic and real scores. The loss is again minimized using backpropagation and gradient descent, only now updating the discriminator.

This alternating optimization of the generator and the conditional discriminator repeats until a termination condition is met, such as a fixed number of iterations or a quantitative measurement of the generator’s performance.

Using the trained model

Once trained, the structure generator (Fig. 8i) may produce structure images of any size with precisely controlled variations (Fig. 8j) in the attribute values. Given that precise control, it is possible to have arbitrarily accurate control by optimizing the input attribute values to minimize the distance between the resulting output’s attribute values and the target attribute values. This process can also correct for some 2D-3D discrepancies, though at higher computational cost from generating and analyzing many 3D samples.

The generator is verified by generating structure images as the final output (Fig. 8k) with attribute values equal to the original segmented structure image(s), then evaluating and comparing other derived attributes of both images, e.g., permeability through a phase. These derived attributes are not part of the training. If the derived attributes are comparable, we consider the generator to have captured the original structures with sufficient accuracy.

After that verification, the trained model can predict what derived attributes a segmented structure image with some input attributes would have. For example, in combination with standard analysis algorithms, the trained model can help answer questions such as “if a formulation had a porosity of 20%, what would its permeability be?” This predictive capability enables searching for a percolation threshold, since the porosity of the generated structure can be precisely reduced until percolation stops, without losing the morphological characteristics of the original structure. These predictions from numerically synthesized image volumes must be validated by analyzing additional real samples. However, it has regardless the potential to provide a more informed basis for that additional analysis than naïve strategies such as grid-search.

Intended meaning of extrapolation

We describe this method as capable of extrapolation because it allows for the generation of large volumes with different attributes from a single image or image volume. However, the amount of extrapolation is defined in terms of, and limited by, the amount of variation in the training data. For example, decomposing a 3D image volume with a porosity of 40% into thousands of small 2D image tiles could yield a training data with a distribution of porosities, e.g., with a mean of 40% and a standard deviation of 5%. The training procedure aims to learn to generate any attributes within that distribution. This means that the morphology of structures with “extrapolated” attributes are still based on real data. It is possible to learn a larger distribution of attributes than the real data contains using the regression classifier approach. However, even with the regression classifier, if a sample exhibits near-zero heterogeneity at a given tile size, then the trained model will exhibit near-zero capability to produce varying attributes.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Author contributions

T.H. designed, implemented and applied generative AI structure synthesis method; C.M., P.Y., P.T., K.S. and K.N. lead tablet study; A.K., S.Z. and M.G. lead implant study; K.N. and S.Z. conceptualized the initial scope; S.Z. lead the design and execution of the project; S.Z., T.H., C.M., K.N., and A.K. wrote the paper.

Peer review

Peer review information

Nature Communications thanks Rydvikha Govender and the other anonymous reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Data availability

All data published in this paper are collected or generated by digiM. Raw XRM data, FIB-SEM data, segmentation data, AI generated data, and quantification data are publicly accessible via a dedicated webpage²⁸ which requires a registration due to the data size and hosting server sustainability and stability considerations. Code, software, models are available upon appropriations of licensing, legal, and hosting agreements. Source data corresponding to the plots are provided as a Source Data file. Source data are provided with this paper.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41467-024-54011-9.