Knowledge-graph-guided diffusion for culturally authentic modernization and animation of heritage patterns

Abstract

Cultural heritage preservation involves critical trade-offs between traditional authenticity and modern commercial utility in transforming historical patterns into animatable digital media. This paper presents a unified framework integrating cultural semantic encoding, diffusion models guided by knowledge graphs, and multi-view animation synthesis to modernize traditional patterns while preserving cultural authenticity. The approach includes a knowledge graph of 487 cultural concepts with 1243 semantic relations to condition the diffusion generation process. Comprehensive experiments on 10,010 culturally diverse patterns achieve a Fréchet Inception Distance of 24.3, representing a 7.6% improvement over state-of-the-art baselines with a Cultural Semantic Similarity of 0.85, exceeding existing methods by 18.1%. Expert evaluation by 50 cultural specialists confirms cultural fidelity scores of 4.6 ± 0.3 on a 5-point scale. Cross-cultural analysis of generalization confirms strong performance on Chinese, Persian, and Islamic traditions with 0.82–0.87 Cultural Semantic Similarity scores. The framework is successful in deploying on varied hardware platforms achieving 10× model compression on mobile devices with negligible quality degradation. These results establish systematic methods for heritage-aware generative systems, demonstrating that culturally explicit constraints enhance creative output while enabling commercially viable applications that respect traditional artistic expression.

Introduction

In today’s digital culture era, preservation has emerged as a central concern, as artificial intelligence technologies offer transformative approaches to the management and revitalization of traditional art forms. Scholarly works demonstrate that combining deep learning with knowledge graphs substantially advances digital cultural heritage management, enabling systematic approaches to automatic documentation as well as semantic comprehension¹. Comprehensive bibliometric reviews reveal that AI is increasingly reshaping museum preservation practices by addressing fundamental challenges in accessibility and conservation². This development calls for sustained research effort, fostering new agendas in digital heritage innovation³. Recent academic studies have identified key trends and persistent challenges in digital cultural heritage research⁴. This trend coincides with growing commercial interest in integrating traditional design elements into contemporary products, necessitating careful approaches to balancing modernization with cultural fidelity.

The significance of heritage preservation extends beyond technical digitization to encompass community identity and intergenerational transmission. Traditional patterns function as visual carriers of collective memory, contributing to cultural cohesion and belonging across generations⁵. However, cultural heritage values remain insufficiently integrated into contemporary planning and design policies, creating disconnection between traditional art forms and modern commercial contexts⁶. Digital technologies offer promising solutions for bridging this gap, though their application requires careful attention to cultural sensitivity and community engagement to ensure that preservation efforts respect the source communities⁷. These considerations highlight the necessity of developing frameworks that address both generation quality and meaningful cultural preservation.

Beyond pattern preservation, artificial intelligence has also been applied to automated visual analysis of material surfaces in related heritage and construction domains. Deep learning methods have been developed for non-invasive defect detection on cultural heritage surfaces directly from images, supporting condition assessment of artefacts and monuments⁸. Comparable approaches have been reported for ordinary buildings, where deep learning models combined with UAV photogrammetry are used to identify exterior cracks⁹, and for large-scale infrastructure, where photogrammetry-based pipelines enable crack digitalization in tunnel and underground structures¹⁰. These studies indicate that visual analysis techniques can be adapted to domain-specific inspection tasks, which is relevant to the present work on generative modelling of cultural patterns, where domain knowledge similarly needs to be incorporated into the visual processing pipeline.

Denoising diffusion probabilistic models have had a profound impact on generative AI, establishing new benchmarks for high-quality image synthesis¹¹. Building on these foundations, latent diffusion models excel at high-resolution image synthesis while substantially reducing computational requirements through latent-space compression¹². Large-scale pretraining combined with advanced conditioning methods has dramatically expanded generative capabilities across domains, ushering in a new era of text-to-image synthesis¹³. In cultural heritage areas, advances in machine learning, especially deep learning, have achieved notable success in areas such as information modelling, 3D reconstruction, and semantic analysis¹⁴. However, transforming traditional patterns into commercially viable animated designs while preserving cultural integrity remains a formidable challenge.

Conditional control mechanisms have become central to directing diffusion processes with greater precision. The optimization of models of guided diffusion increases sampling efficiency with quality preservation¹⁵, while newer architectures aid in greater semantic alignment in conditional text-to-image synthesis¹⁶. Comprehensive surveys highlight growing applications while underscoring persistent challenges in fine-grained attribute control and stylistic consistency¹⁷. Reinforcement learning approaches address diversity limitations in synthetic image generation¹⁸. The advent of ControlNet represented a critical breakthrough with its presentation of spatial conditioning controls that can be added seamlessly to pretrained models¹⁹, with later improvements boosting efficiency through consistency feedback mechanisms²⁰ and architectural optimization²¹. Existing frameworks, however, lack dedicated mechanisms for culturally meaningful preservation in heritage pattern modernization.

The style transfer methods developed from optimization-based convolutional networks²² to perceptual loss functions allowing real-time processing²³ up to advanced diffusion-integrated methods²⁴. Current methods take advantage of several references in achieving photorealistic style transfer in modernizing historical images²⁵, while bidirectional unsupervised transformations conserve semantic content while stylizing attributes²⁶. Current developments further extend applications to three-dimensional neural stylization²⁷ and field-specific applications such as architectural photo-taking²⁸. Existing methods, however, are ill-equipped to address the specific requirements of cultural pattern modernization, particularly the preservation of symbolic meanings and structurally significant motifs.

Texture synthesis and animation creation are mutually orthogonal research areas that inform approaches to modernizing patterns. Sophisticated methods enable advanced texture synthesis on complex geometries with editable neural radiance fields²⁹, guided correspondence flows³⁰, and long-range restrictions that enable high-resolution synthesis³¹. Customized implementations guarantee view-consistent texture creation with geometric accuracy³², with diffusion-based methods providing improved control with inversion methods³³. Animation has advanced drastically, with multi-view video diffusion being used to enable coherent 3D model animations³⁴, in addition to extant frameworks that chronicle rapid developments as well as the future challenges³⁵. Furthermore, diffusion transformers are proficient in spatiotemporal dependency capture³⁶, with latent diffusion models obtaining high-resolution time synthesis with hierarchical methods³⁷. Point cloud processing methods highlight culture-informed computational approaches that include semantic segmentation of cultural heritage locations³⁸, deep learning method comparison for 3D heritage segmentation³⁹, archaeological site identification through LiDAR⁴⁰, as well as machine learning approaches that aim to detect latent archaeological features¹². Recent developments have started addressing culture-informed content creation with text-guided icon creation¹⁹ as well as subject-centric creation through fine-tuning¹, however, these are mostly centered on stills with complete pattern modernization pipelines remaining untouched. Table 1 presents a synthesis of existing approaches and their respective limitations that motivate this study.

Table 1.

Summary of state-of-the-art methods and identified research gaps.

Research domain	Representative methods	Limitations	Gap addressed in this study
Diffusion models	DDPM11, Stable Diffusion12, ControlNet19	Spatial control without cultural semantic constraints	Knowledge-graph conditioning for cultural preservation
Style transfer	Gatys et al.22, Johnson et al.23, Zhang et al.24	Content-style separation loses symbolic meanings	Rule-based cultural constraint integration
Heritage digitization	Point cloud segmentation36, HBIM workflows14	Primarily 3D geometric reconstruction	2D pattern generation with animation capability
Video generation	Multi-view diffusion34, Video transformers33	General temporal coherence without cultural motion semantics	Culturally-appropriate animation synthesis

In spite of significant advancements in these areas, critical gaps persist in cultural heritage pattern modernization. Existing systems lack specialized encodings for cultural semantics and symbolic associations. Current models lack adequate detail for preservation of symbolic integrity under transformations. Integration with unified commercial frameworks remains largely unexplored.

The current moment presents a timely opportunity to address these gaps. The convergence of large-scale pretrained diffusion models, openly accessible heritage digitisation initiatives such as the Metropolitan Museum Open Access collection, and growing institutional demand for culturally informed digital media creates favourable conditions for systematic framework development. Moreover, the increasing commercial interest in culturally themed products across fashion, gaming, and interactive media industries underscores the practical urgency of bridging technical generation capabilities with culturally grounded constraints.

This study presents a framework built upon three core innovations: cultural semantic encoding that maintains symbolic integrity, guided diffusion that integrates cultural constraints, and commercially deployable animated reconstruction. In-depth experiments confirm its performance across a variety of assorted datasets and realistic applications, providing systematic methods for converting heritage patterns to commercially consumable assets that retain their cultural meaning.

Methodology

Overview of the framework

The proposed system establishes an end-to-end pipeline that transforms traditional cultural patterns into commercially deployable digital animations while maintaining cultural integrity, as illustrated in Fig. 1. The system design consists of five interrelated modules that sequentially apply a series of stages to input patterns: encoding, generation, animation, and optimization stages.

Fig. 1.

Overview of the proposed framework architecture with five core processing modules and iterative feedback mechanism.

The framework takes as input traditional cultural patterns with semantic annotations. The inputs are then subjected to the sub-module of Cultural Semantic Encoding that obtains features along with structural constraints that are relevant to culture. The encoded forms are used as conditioning signals for the sub-module of Diffusion-Based Generation that makes use of latent diffusion models¹², with spatiotemporal control mechanisms synchronized with them, to create modernized patterns. The new patterns preserve their symbolic content while possessing modern aesthetic features. The patterns statically generated are output as sequences of time by the sub-module of Animated Reconstruction with multi-view diffusion methods used to create coherent patterns with culturally appropriate motion features that are animated. Finally, the Commercial Optimization sub-module performs format conversion, quality enhancement, and deployment preparation to ensure outputs conform to industry standards across commercial applications including digital media, fashion design, and interactive installations.

Central to this approach is an iterative refinement cycle implemented as a feedback loop, enabling continuous quality improvement through multi-stage validation against both cultural preservation criteria and commercial viability requirements.

Cultural semantic encoding

The cultural semantic encoding module transforms traditional patterns to semantically-enriched representations that preserve cultural integrity as indicated in Fig. 2. The module integrates visual feature extraction with knowledge-led semantic encoding to extract both low-level visual features as well as high-level cultural semantics.

Fig. 2.

Cultural semantic encoding module architecture.

The encoding process initiates with visual feature extraction through a hybrid architecture combining CNNs and transformers. Given input pattern , the visual encoder produces multi-scale features:

1

where CNN extracts spatial features, Trans models global dependencies, and Pool aggregates multi-scale representations into feature vector .

Semantic encoder involves conditional control processes¹⁹ that are applied to cultural preservation initiatives. The knowledge graph G provides structured cultural concepts as per digital preservation of cultural heritage processes¹. The encoder utilizes three joint operations. Firstly, knowledge-guided attention aligns visual features with cultural semantics:

2

where maps graph concepts to embedding space. Second, cultural attention identifies semantically significant elements:

3

Third, constraint integration enforces cultural rules K including color symbolism and compositional principles through rule-based filtering:

4

The final encoding combines these components through weighted fusion and normalization:

5

where with serves as the cultural semantic representation. Training optimizes reconstruction quality, semantic consistency, and cultural preservation through multi-objective loss:

6

This encoding maintains cultural fidelity while enabling subsequent generation processes.

Diffusion-based pattern generation

Pattern synthesis is performed with denoising diffusion probabilistic models¹¹ to create culturally plausible traditional patterns according to directives of semantic encodings introduced in “Cultural semantic encoding” section, as demonstrated in Fig. 3. Unlike conventional generative adversarial networks, diffusion models can produce better synthesis quality with iterative refinement settings⁴¹, thus rendering them highly proficient in preserving complex cultural patterns inscribed in traditional patterns.

Fig. 3.

Diffusion-based pattern generation architecture with U-Net backbone and cultural semantic conditioning.

The generation architecture adopts a U-Net backbone with multi-scale encoder-decoder structure. Given noisy pattern at timestep t ∈ [0,T], the forward diffusion process gradually adds Gaussian noise following a variance schedule . The denoising network predicts noise to be removed at each step, conditioned on both timestep embedding and cultural semantic encoding :

7

U-Net encoder obtains multi-scale features through progressive downsampling, while pattern reconstruction in decoder is through upsampling with skip connections that preserve fine-grained details. The time embedding transfers discrete timesteps to continuous representations, condition fusion mechanisms inject cultural semantics with cross-attention layers. Stable gradient flow is preserved through residual blocks used with self-attention in bottle-neck layer to capture overall pattern coherence.

For efficient generation, the reverse sampling follows DDIM formulation⁴²:

8

The training target integrates plain noise prediction loss with perceptual quality measures. We utilize LPIPS distance⁴³ as a perceptual similarity measurement between generated patterns and target patterns. To address high-frequency details essential for conventional patterns, we add loss functions suitable for extreme cases of super-resolution⁴⁴. In addition, shift robustness-based similarity measures⁴⁵ are used to preserve spatial coherence, while texture synthesis methods⁴⁶ conserve fine pattern details at the grain level during pattern synthesis. The framework facilitates controllable pattern upgradation with cultural authenticity while accommodating different applications such as style transfer²⁴ as well as pattern variation synthesis.

Animated reconstruction

The reconstructed animation module converts these 2D patterns developed to 3D sequences that can be visualized immersively and used interactively as shown in Fig. 4. The developed approach addresses the challenge of retaining cultural authenticity while introducing temporal dynamics with multi-view consistency and control of motion.

Fig. 4.

Animated reconstruction pipeline from 2D patterns to 3D animation sequences.

The reconstruction pipeline initiates with depth and geometry estimation from 2D patterns. Given input pattern , monocular depth estimation predicts depth map D:

9

Surface normal prediction recovers geometric orientations, enabling 3D mesh construction that maintains pattern integrity during transformation.

Multi-view consistency ensures that the appearance is preserved across view orientations. According to multi-view video diffusion approaches³², cross-view attention maintains spatial consistency. The multi-view generation target minimizes view inconsistency:

10

where represents view i and denotes geometric warping between views. This approach generates geometrically consistent patterns from multiple viewpoints simultaneously.

Temporal variations are represented from video diffusion models’ perspectives. With the aid of sophisticated deep video synthesis tools^33,47, time coherence is maintained through:

11

where denotes frame f at diffusion timestep t, provides temporal context, and c represents conditioning including cultural semantics and motion control parameters. The motion control modules help to accurately fine-tune animation settings so that it becomes possible to transition smoothly while maintaining cultural attributes.

For purposes of high-resolution synthesis, latent diffusion models³⁴ are simulated in compressed spaces of representation. The overall optimization comprises reconstruction, multi-view, as well as temporal terms:

12

This approach achieves high-resolution video synthesis while simultaneously maintaining temporal coherence and multi-view consistency, making it well suited for cultural heritage visualization, educational displays, and interactive exhibitions where dynamic presentation enhances pattern appreciation.

Commercial application optimization

Practical commercial deployment requires optimization for mobile platforms and real-time usage without compromising generation quality. This section examines the computational performance and resource constraints essential for broader adoption across design studios, cultural organizations, and consumer software.

Large-scale model compression that preserves cultural fidelity is essential for on-device deployment. Borrowing from state-of-the-art developments on on-device style transfer⁴⁸, the system employs neural architecture search to uncover light-weight network topologies deployable on mobiles. This form of compression drastically reduces model parameters while holding capacity for semantic encoding with distillation of knowledge. The compressed encoder satisfies memory constraints while maintaining feature similarity:

13

where represents the original encoder and measures distribution divergence between teacher and student predictions.

Diffusion acceleration employs guided distillation procedures¹⁵ to reduce sampling steps from hundreds to tens without sacrificing generation fidelity. Progressive distillation learns to predict multiple steps ahead through

14

where k > 1 is compatible with multi-step predictions. This approach achieves 10 × speedup to enable real-time pattern design on consumer-grade hardware. The compressed models retain cultural conditioning functionality such that generated patterns retain authenticity despite computational constraints.

Latency optimization utilizes a mix of mixed-precision inference along with dynamic batching techniques. Through critical path analysis, it detects bottleneck operations within the generation pipeline such that certain optimization can be taken through operator fusion along with memory access pattern optimization. The framework achieves sub-second generation times on mobile GPUs while maintaining perceptual quality with carefully tuned distillation objectives that privilege cultural semantic preservation over pixel-accurate reconstruction.

These optimizations enable deployment on a range of commercial applications including mobile design tools, real-time exhibition displays, and interactive cultural education platforms. These optimizations broaden access to traditional pattern modernization while ensuring computational feasibility across hardware configurations ranging from high-end workstations to resource-constrained mobile devices.

Experimental evaluation

Experimental setup

This research tests the proposed framework on publicly accessible traditional pattern datasets in order to guarantee reproducibility. There are three respective datasets with varied cultural depictions: Metropolitan Museum of Art Open Access collection (2847 high-quality patterns from Chinese, Persian, and Islamic styles), Describable Textures Dataset (5640 texture images for geometric analysis), and WikiArt Decorative subset (1523 pieces of ornamental art). Specifications can be seen in detail in Table 2. All images are preprocessed as 512 × 512 pixels with 80/10/10 train/validation/test divides. Random horizontal flips, rotations (+ /− 15°), as well as color jittering are used as part of the data augmentation in order to retain structural features⁴⁹.

Table 2.

Dataset specifications.

Dataset	Source	Images	Cultural origins	Split (Train/Val/Test)
Met Museum Open Access	Metropolitan Museum	2847	Chinese, Persian, Islamic	2278/285/284
DTD	University of Oxford	5640	Pattern textures	4512/564/564
WikiArt Decorative	WikiArt Project	1523	Multi-cultural	1218/153/152
Total	Combined	10,010	Diverse traditions	8008/1002/1000

Significant values are in [bold].

Framework is trained in PyTorch 2.0 on 4 × NVIDIA A100 GPUs with 40 GB memory. The cultural semantic encoder is trained on pretrained ResNet-50 on ImageNet with 1 × 10⁻⁴ learning rate. The diffusion model is U-Net⁵⁰-type architecture with 256 base channels trained for 500,000 iterations with AdamW optimizer (β₁ = 0.9, β₂ = 0.999, 2 × 10⁻⁵ learning rate). Diffusion is trained with 1000 timesteps and inferred with 50 DDIM steps. The knowledge graph contains 487 cultural concepts with 1243 semantic relations. It would take 72 h to complete training with mixed-precision turned on.

Comprehensive assessment utilizes several metrics for cultural fidelity and generation quality. Fréchet Inception Distance (FID) estimates distributional similarity⁴⁹, while Inception Score (IS) assesses diversity⁵⁰. Learned Perceptual Image Patch Similarity (LPIPS) approximates perceptual quality with deep features⁴³. When approximating animations, Fréchet Video Distance (FVD) approximates temporal coherence⁵¹. The paper contributes Cultural Semantic Similarity (CSS), approximating adherence to cultural constraints with knowledge graph embeddings.

The CSS metric quantifies cultural semantic preservation using knowledge graph embeddings. The semantic encoder extracts concept activation vectors from both input and generated patterns, identifying the top-k concepts with highest attention weights from the knowledge graph. Each concept is represented by a 256-dimensional embedding vector obtained through TransE training. Given input concept set and generated concept set , where each concept has embedding and normalized attention weight , CSS is computed as:

15

where denotes cosine similarity and . The parameter k is set to 10 empirically. Scores range from 0 to 1, with higher values indicating stronger cultural semantic preservation.

Baselines are also shown as StyleGAN3, Stable Diffusion 2.1, ControlNet, pattern synthesis with GANs, and also with neural style transfer. The overall experimental setup is summarized in Table 3.

Table 3.

Experimental configuration.

Component	Specification
Hardware	4 × NVIDIA A100 GPU (40 GB)
Framework	PyTorch 2.0, CUDA 11.8
Training	500 K iterations (~ 72 h)
Encoder	ResNet-50, LR: 1 × 10−4
Diffusion	U-Net (256 channels), LR: 2 × 10−5
Batch Size	Train: 64, Inference: 16
Optimizer	AdamW (β1 = 0.9, β2 = 0.999)
Timesteps	Train: 1000, Inference: 50 (DDIM)
Knowledge graph	487 concepts, 1,243 relations
Metrics	FID↓, IS↑, LPIPS↓, FVD↓, CSS↑

The architectural characteristics of the tested patterns vary across cultural origins. Chinese geometric patterns typically exhibit fourfold rotational symmetry with 6–10 distinct motif elements per unit. Persian carpet designs demonstrate bilateral symmetry organized around central medallions. Islamic patterns feature girih-based tessellations with 4–7 polygon types per repeating unit. These structural differences influence generation difficulty, with higher element counts correlating with increased FID variance.

Quantitative results

This study conducts a comprehensive quantitative evaluation using multiple metrics to assess the proposed framework’s performance relative to traditional pattern synthesis as well as its capacity to preserve cultural semantics. Performances are compared with existing metrics such as Fréchet Inception Distance (FID), Inception Score (IS), Learned Perceptual Image Patch Similarity (LPIPS), as well as a new metric referred to as Cultural Semantic Similarity (CSS). The new metric is constructed to verify whether cultural preservation is well-maintained.

As shown in Fig. 5a, the proposed framework consistently outperforms all five state-of-the-art baselines across evaluation metrics. For visual comparison, all metrics are normalized to theoretical reference ranges that represent typical performance bounds in the literature (FID: 20-50, IS: 3.0-4.5, LPIPS: 0.10-0.30, CSS: 0.30-1.0), enabling fair cross-metric comparison. The detailed quantitative results are presented in Table 4. The proposed framework achieves an FID of 24.3, a 7.6% improvement over the best-performing baseline, Stable Diffusion 2.1 (FID: 26.3). The framework achieves an IS of 3.95 with a marginal gain over Stable Diffusion’s 3.85, while obtaining the minimal value of 0.158 on the LPIPS score compared with its closest rival’s 0.165 score. Most notably, the CSS metric demonstrates that our framework significantly exceeds its strongest competitor in cultural semantic preservation with a score of 0.85 compared with ControlNet’s 0.72, accounting for an 18.1% improvement, thereby validating the effectiveness of the knowledge-graph-guided cultural constraints. Traditional GAN and Neural Style Transfer methods show considerably lower normalized scores across all metrics, indicating their limited capability in handling culturally constrained pattern generation tasks.

Fig. 5.

Quantitative performance evaluation.

Table 4.

Quantitative performance comparison.

Method	FID↓	IS↑	LPIPS↓	CSS↑
StyleGAN3	28.5	3.72	0.182	0.56
Stable diffusion 2.1	26.3	3.85	0.165	0.68
ControlNet	27.8	3.68	0.178	0.72
Traditional GAN	42.1	3.21	0.245	0.42
Neural style transfer	38.7	3.34	0.238	0.38
Proposed framework	24.3	3.95	0.158	0.85

Lower values are better for FID and LPIPS; higher values are better for IS and CSS.

Significant values are in [bold].

The complete comparison in Fig. 5b provides detailed statistics for three representative methods across different evaluation metrics. Under the theoretical reference range normalization, our framework achieves normalized scores of 86% for FID, 63% for IS, 71% for LPIPS, and 79% for CSS, consistently outperforming both Stable Diffusion and ControlNet across all metrics. The relatively lower normalized IS score reflects the challenging nature of achieving diversity in culturally constrained generation, while the strong FID performance demonstrates superior image quality.

The training convergence curves in Fig. 5c indicate stable optimization across all loss terms over 500,000 iterations. The total loss decreases from 2.5 to 0.3, while the convergences of individual loss terms, namely reconstruction loss, semantic loss, and cultural loss, follow different patterns over optimization due to their respective complexities. In particular, cultural loss converges with the lowest speed, which is a result that is compatible with the complex nature of cultural semantic constraints imposed from knowledge graphs.

Cross-cultural generalization, a prerequisite for successful deployment, is evaluated across four distinct cultural groups as can be seen from Fig. 5d and analyzed in Table 5. The model is seen to perform well across Chinese, Persian, Islamic, and mixed cultural patterns with 0.82–0.87 performance on the CSS while simultaneously exhibiting constant preservation of culture fidelity across varied origins. The marginally improved performance within Chinese patterns (CSS: 0.87) can be explained through a distribution of the training data such that perpetual strong generalization within diversified cultural settings is made feasible again. Collectively, these findings confirm the model’s capacity to generate high-quality traditional patterns that retain authentic cultural properties across a diverse range of origins.

Table 5.

Performance across cultural types.

Cultural type	FID↓	CSS↑	Samples
Chinese	23.8	0.87	1200
Persian	25.1	0.82	850
Islamic	24.7	0.84	950
Mixed	24.0	0.85	1000
Average	24.4	0.85	1000

Consistent performance demonstrates robust cross-cultural generalization.

Significant values are in [bold].

Qualitative analysis

The critical visual analysis dictates the pattern design methods’ ability to maintain cultural authenticity. The qualitative analysis exhibited in Fig. 6 portrays diversified performance profiles among four illustrative cultural groups: Chinese geometric patterns, Persian carpet rugs, Islamic Mihrab pattern designs, and sophisticated interlace panels. Careful examination reveals that our system performs better than baselines in maintaining culturally appropriate visual content. StyleGAN3 exhibits significant color distortion and compositional changes, obvious in Chinese geometric patterns, which produce unfamiliar pink-red hues instead of traditional colors nearer to the ground. Moreover, Persian carpets no longer possess their unique deep burgundy shade. The sharp clarity developed using Stable Diffusion 2.1 is satisfactory in general but includes too much blur that makes subtle cultural content invisible, for example, the Islamic Mihrab pattern design. Therefore, subtle features within calligraphy, along with some geometric patterns, are hard to see. ControlNet creates overall forms by using the effects of space. It does not make precise copies of texture or color, so the patterns keep their form while being quite different from the correct cultural styles.

Fig. 6.

Qualitative comparison of different methods for cultural pattern generation across four cultural categories.

Figure 7a shows that the model is very good at keeping complex cultural details at a small scale with specific local maps. Traditional bloom patterns, geometric shapes, and designs show better edge clarity and color accuracy in the results of the new method compared to the Stable Diffusion benchmark, where similar areas appear blurrier and less colorful. The investigation on generative diversity from Fig. 7b confirms the framework’s ability for controllable variability, such that it always yields different stylistic variations while retaining cultural semantic consistency across all generated specimens. This property is especially valuable for commercial applications that demand several design variations within culturally fitting limits.

Fig. 7.

Supplementary analysis of cultural pattern generation performance.

The user study results presented in Fig. 7c and tabulated in Table 6 give a quantitative validation of the qualitative results obtained from an expert survey with 50 cultural heritage experts. It can be perceived from Table 6 that our approach obtains a high cultural fidelity score of 4.6 ± 0.3 on a 5-point scale, significantly surpassing StyleGAN3 (2.3 ± 0.5), Stable Diffusion (3.1 ± 0.6), and ControlNet (3.5 ± 0.4). The scores on visual quality also trail similarly encouraging results with a score of 4.5 ± 0.4 that reflects a well-balanced performance in both aspects of cultural preservation as well as visual quality. Overall preference distributions suggest that 38% of experts are in favor of that proposed method, while 25% are for Stable Diffusion, 22% are for ControlNet, and 15% are for StyleGAN3 only, thus substantiating that method’s practical supremacy for cultural pattern modernization applications. Furthermore, a statistical significance test discloses that the benefits of such a proposed method as cultural fidelity, are highly significant (p < 0.001), which supports the method’s cultural semantic encoding as well as guided generation functionalities effectively.

Table 6.

User study results comparing cultural fidelity and visual quality.

Method	Cultural fidelity (1–5)	Visual quality (1–5)	Overall preference (%)
StyleGAN3	2.3 ± 0.5	3.8 ± 0.4	15
Stable diffusion 2.1	3.1 ± 0.6	4.2 ± 0.3	25
ControlNet	3.5 ± 0.4	4.0 ± 0.5	22
Ours	4.6 ± 0.3	4.5 ± 0.4	38

Significant values are in [bold].

Ablation study

Systematic ablation experiments precisely assess the impact of individual framework elements with controlled ablation experiments as presented in Fig. 8 and in summary in Table 7. The experimental method keeps balanced training settings with intentional disabling of certain modules to determine their performance impact on four test measures: FID, Inception Score, LPIPS, and Cultural Semantic Score.

Fig. 8.

Ablation study analyzing the impact of individual framework components on generation performance across four evaluation metrics.

Table 7.

Ablation study quantitative results.

Variant	FID ↓	IS ↑	LPIPS ↓	CSS ↑
Full model (Ours)	24.3	3.95	0.158	0.85
w/o Cultural Semantic Encoding	29.8	3.58	0.195	0.64
w/o Knowledge Graph	27.5	3.72	0.178	0.71
w/o Cultural Attention	26.8	3.78	0.172	0.74
w/o Constraint Integration	26.2	3.82	0.168	0.77
w/o Multi-view Consistency	25.7	3.86	0.163	0.80
w/o Perceptual Loss (LPIPS)	25.1	3.88	0.192	0.81
Baseline (Only L2 Loss)	31.2	3.42	0.218	0.58

↓ indicates lower is better, ↑ indicates higher is better. Bold values represent the best performance achieved by the full model.

The figure corresponding to Fig. 8a–d reveals that the most critical component of the framework is the Cultural Semantic Encoding module. If we disable it, then the FID increases drastically from 24.3 to 29.8, as well as the score for Cultural Semantic from 0.85 down to 0.64. This brings down the capacity to retain culture by 24.7%. The dramatic performance drop reveals how critical it is to integrate cultural knowledge graphs with semantic attention techniques. It preserves significant amounts of real cultural features while forming patterns. Furthermore, the Cultural Semantic Encoding contributes 20.0% of the overall system performance compared to other individual components. This is indicated in Fig. 8f, impact ranking analysis.

The combination of the knowledge graph, along with cultural attention mechanisms, is used to reinforce one another as part of semantics preservation. When disabling the knowledge graph module, the FID rises to 27.5 while the CSS drops to 0.71, showing that structured cultural knowledge makes a significant impact in boosting up the model’s understanding of traditional design principles. In a similar fashion, when we disable the cultural attention mechanisms, we see significant performance drops (FID: 26.8, CSS: 0.74), which supports selectively highlighting culturally educative visual features during encoding.

The perceptual loss part reveals a very distinguishable feature during ablation analysis. As shown in Fig. 8c, removal of perceptual optimization based on LPIPS leads to a rise in the value of the LPIPS metric itself, from 0.158 up to 0.192. This undoubtedly validates that the perceptual loss indeed reduces the perceptual gap between generated patterns as well as genuine ones. Such an inverse relationship reinforces the theoretical basis for integrating perceptual metrics into the training objective, ultimately aimed at improving visual quality.

Multi-view consistency and constraint integration modules add up to deliver moderate but significant performance increments. Multi-view consistency decreases FID by 1.4 points (25.7 compared to 24.3), while constraint integration adds 1.9 points of gain. The base setup that uses single-mode L2 reconstruction loss alone yields the worst performance on all measures (FID: 31.2, IS: 3.42, CSS: 0.58), which shows a 27.9% overall performance gap with respect to the complete framework and verifies the additive advantage of our proposed multi-component framework.

Commercial application cases

Experimental deployment cases clearly substantiate the framework’s commercial applicability with a comprehensive performance analysis over a very large number of hardware platforms ranging from high-end desktop PCs to computationally limited edge devices. As can be seen from Fig. 9a, analysis of the quality-latency trade-off reveals characteristic performance trends over six varied deployment configurations. The desktop GPU deployment obtains maximum performance with a peak FID score of 24.3 when its inference latency is 45 ms, well within its optimal operating region where it balances generation quality with computational throughput. In contrast, mobile high-end devices allow only 2.5 FID points of quality reduction while still achieving sub-second times of generation at 320 ms of latency, thus affirming the superiority of proposed model compressing methods even under computationally limited settings.

Fig. 9.

Commercial deployment performance analysis across six hardware platforms.

The compression analysis in Fig. 9c verifies successful achievement of the 10 × compression target mentioned in the methodology part. Using progressive distillation as well as neural architecture search, the mid-range mobile deployment compresses the model from 450 MB down to 45 MB, achieving precisely the desired compression target specified by the reference point threshold. This substantial size reduction makes it feasible to deploy on consumer mobile devices while maintaining satisfactory generation quality (FID = 28.5), which is only a 4.2-point drop off from the full desktop model configuration. In comparison, the edge device deployment achieves up to 14.1 × compression ratio, obtaining 180 MB memory footprint as can be seen from Fig. 9b, but it does so at a quality degradation penalty of achieving FID = 29.8.

Commercial viability analysis using user satisfaction analysis, presented in Table 8 and Fig. 9d, confirms the acceptability of the framework under various deployment situations. The expert evaluation recruited 50 participants: 22 academic researchers in traditional art studies, 16 museum professionals from northeastern China, and 12 traditional craft practitioners. Participants evaluated 40 randomly assigned pattern pairs on four dimensions (visual authenticity, symbolic accuracy, compositional appropriateness, overall fidelity) using 5-point scales. Patterns were presented without method labels in randomized order. Inter-rater agreement assessed by Krippendorff’s alpha reached 0.73, indicating moderate consistency. Scores ranged from 3.5 to 4.7 across platforms, with desktop configurations receiving higher ratings than mobile deployments.

Table 8.

Performance across deployment platforms.

Platform	Latency (ms)	Memory (MB)	Quality (FID)	Model size (MB)	User score (1–5)
Desktop GPU	45	1200	24.3	450	4.7
Workstation	78	980	24.8	385	4.5
Mobile High-end	320	450	26.8	85	4.2
Mobile Mid-range	680	280	28.5	45	3.8
Web Browser	520	320	27.2	65	4.0
Edge Device	890	180	29.8	32	3.5

Lower values are better for Latency, Memory, FID, and Model Size. Higher values are better for User Score. Desktop GPU maintains full model quality (FID = 24.3). Mobile Mid-range achieves 10 × compression target with acceptable quality degradation. 4 out of 6 platforms achieve user satisfaction ≥ 4.0

Significant values are in [bold].

The desktop and workstation platforms attain high acceptability levels (4.5–4.7), while mobile high-end deployment records good acceptability at 4.2, which is well above the stated acceptability threshold of 4.0. The analysis shows that 4 of 6 deployment platforms are above this acceptability threshold, thus substantiating acceptability of the framework for varied commercial applications such as professional design studios, mobile creative applications, museum interactive displays, as well as web-based cultural education platforms. The resources consumption analysis shows successful accommodation under different computational limitations with memory consumption ranging from 1200 MB for desktop deployment to 180 MB for edge devices, thereby allowing for broad accessibility while retaining core cultural pattern creation functionality across the performance range.

Ethics approval and consent to participate

The user evaluation component of this study involved human participants for the assessment of cultural fidelity and visual quality of generated heritage patterns. All methods were carried out in accordance with relevant guidelines and regulations, including the Declaration of Helsinki (as revised in 2013). The experimental protocol was reviewed and approved by the Ethics Review Committee of Beihua University (Approval No. BHU-EC-2025-021; Approval Date: February 24, 2025). All participants were adults aged 18 years or older. Written informed consent was obtained from all individual participants and/or their legal guardians, as applicable, prior to the evaluation session. As the study did not involve minors or individuals lacking legal capacity, consent from legal guardians was not required in practice. No personally identifiable information was recorded, and all rating data were stored in de-identified form.

Discussion

Analysis of results

Experimental results demonstrate that the proposed framework surpasses conventional methods in revitalizing cultural heritage patterns, owing to its novel integration of cultural semantic encoding, guided diffusion, and animated reconstruction. The framework effectively preserves cultural authenticity while producing commercially viable designs, a capability enabled by its knowledge-graph-guided architecture—in contrast to conventional diffusion methods that typically disregard cultural constraints^11,12. Its FID of 24.3 represents a clear improvement over competing methods, confirming that culturally integrated approaches not only enhance generation quality but also preserve semantic meaning beyond spatially controlled methods¹⁹.

The performance of 0.85 in Cultural Semantic Similarity is greater than ControlNet’s 0.72 performance. This reveals one of the main advantages for employing structured cultural knowledge in the diffusion framework. The performance gap reveals that existing approaches lack complete coverage of cultural meanings and natural symbolic meanings with conventional patterns^1,3. Embedding semantic inter-relations among cultural concepts is central to the knowledge graph, enabling the diffusion process to explore culturally appropriate design alternatives at each denoising step. This directly addresses deficiencies encountered in prior heritage digitalization efforts, where machine learning implementations have otherwise compromised semantic preservation during reconstruction¹⁴.

Compared to traditional style transfer technologies, distinct architectural differences are introduced. While deep network technologies are capable of effectively separating content from style codes^22,23, these technologies involve challenges when converting old patterns to modern styles with cultural authenticity efforts. Diffusion-augmentation style transfer technologies produce improved quality output²⁴, whereas self-learning technologies can produce false alterations when applied to culturally significant designs²⁶. The cultural rules embedded in this system assist in correcting these issues through using knowledge rather than being solely dependent on overall statistical patterns.

Cross-cultural generalization demonstrates robust performance across diverse pattern traditions, with CSS scores ranging from 0.82 to 0.87. The animation module effectively synthesizes temporally coherent sequences using multi-view diffusion³⁴, however, current implementations lack motion semantic controls required to support culturally appropriate dynamic expressions^35,37. The system demonstrates computational efficiency that is appropriate for commercial applications,however, mobile platforms exhibit reduced quality with trade-offs in accessibility over fidelity that are needed to optimize further.

Failure cases and limitations

Despite these contributions, this study records a number of limitations. Representative failure cases are illustrated in Fig. 10, with occurrence rates summarized in Table 9. For Islamic geometric designs containing densely interlaced polygon arrangements, the model occasionally merges adjacent star and polygon elements during denoising, reducing visual clarity in approximately 12% of high-complexity samples (Fig. 10a). Increasing U-Net base channels to 512 offers potential mitigation at the cost of computational overhead. Color drift represents another observed failure mode, where culturally symbolic hues shift toward neighboring tones. Traditional vermillion in East Asian patterns shifted toward orange in roughly 7% of outputs (Fig. 10b), while Persian lapis blue exhibited similar drift in approximately 5% of cases. Strengthening color constraint nodes in the knowledge graph could address this limitation. The animation module produces visible seams at viewing angles exceeding 70 degrees, affecting approximately 8% of sequences (Fig. 10c), particularly on mobile deployments with compressed models.

Fig. 10.

Representative failure cases observed in pattern generation and animation. Red annotations highlight affected regions, including merged elements, color-shifted areas, and visible seams.

Table 9.

Summary of observed failure cases.

Failure type	Affected patterns	Rate	Potential mitigation
Element merging	Islamic geometric (high polygon density)	~ 12%	Increase U-Net channels to 512
Color drift (vermillion → orange)	East Asian floral patterns	~ 7%	Strengthen color constraint nodes
Color drift (lapis blue)	Persian patterns	~ 5%	Strengthen color constraint nodes
View discontinuity	Animation sequences (> 70° angles)	~ 8%	Extend angular consistency loss

The system is currently reliant on manually crafted knowledge graphs with 487 cultural concepts that might not exhaustively reflect the international diversity of heritage, as pointed out in the latest bibliometric surveys on digital cultural heritage^2,4. The current evaluation focuses on Asian and Middle Eastern traditions without systematic testing on European motifs such as Gothic tracery or Celtic knotwork, reflecting dataset availability rather than architectural limitations. To assess the framework’s potential extensibility, we conducted a preliminary structural analysis of three European pattern types sourced from the Victoria and Albert Museum digital collection: Byzantine mosaic tessellations (n = 45), Art Nouveau floral ornaments (n = 38), and Celtic knotwork panels (n = 29). Visual inspection and motif complexity analysis indicate that these patterns share structural properties, including repetitive geometric units and hierarchical symmetry, with the Islamic and Chinese patterns on which the framework was trained. The knowledge graph architecture supports concept node addition without retraining the full model, suggesting that incorporating European cultural concepts such as interlace topology and acanthus symbolism is architecturally feasible. Full validation on European datasets remains a priority for future work, with Europeana and the V&A Open Access collection providing viable data sources.

The experimental comparison does not include Heritage Building Information Modelling (HBIM) workflows as a baseline, as HBIM systems such as those reviewed by Gil et al.¹⁴ primarily address three-dimensional architectural documentation and semantic building element classification, whereas the proposed framework targets two-dimensional surface pattern generation and animation. These represent fundamentally distinct methodological paradigms operating on different input modalities and evaluation criteria: HBIM systems are assessed through geometric accuracy and IFC compliance, while generative pattern frameworks are evaluated through perceptual quality metrics such as FID and cultural fidelity measures such as CSS. Direct quantitative comparison between these paradigms is therefore methodologically inappropriate, and standardized cross-paradigm evaluation protocols remain undeveloped in the field. Future work may explore integration points where HBIM-extracted surface geometry serves as structural conditioning input for the proposed generation pipeline. The expert evaluation, while providing a valuable cultural fidelity assessment, relies on subjective judgment from a regionally concentrated participant pool, which may limit the generalizability of the findings. The 72-h training limitation prohibits fast adaptation to newly registered museums. Future work would be worthwhile in terms of automating knowledge graph crafting from heritage datasets with text-guided synthesis approaches for greater creative control^1,13,19, as well as point cloud processing approaches to three-dimensional reconstruction of cultural patterns³⁸. Few-shot adaptation approaches would be capable of resource-efficient customization to specific cultural settings with preservation of semantic integrity across diversely deployed heritage applications.

Conclusion

This study presents a framework that integrates knowledge-graph-guided diffusion with animated reconstruction for heritage pattern modernization. The approach incorporates structured cultural knowledge into the diffusion process, enabling preservation of symbolic meanings during transformation. The proposed Cultural Semantic Similarity metric provides a quantitative evaluation of cultural preservation that pixel-based metrics cannot capture. Unlike existing diffusion-based methods such as ControlNet, which apply spatial conditioning without cultural semantic awareness, and style transfer approaches that separate content from style at the cost of symbolic integrity, the proposed methodology addresses these critical gaps by integrating cultural semantic encoding, spatial diffusion generation control, and temporal animation synthesis within a coherent end-to-end pipeline. Extensive experiments over 10,010 varied cultural patterns confirm that the framework obtains unprecedented performance with FID of 24.3, a 7.6% improvement over state-of-the-art baselines. Furthermore, a Cultural Semantic Similarity of 0.85 surpasses existing methods by 18.1%, validating the effectiveness of structured cultural knowledge injection in preserving symbolic integrity during pattern modernization.

The knowledge graph is designed with 487 cultural concepts and 1243 semantic links. In this configuration, it generalizes well across varied cultures and thus performs well on Chinese-rich datasets, along with datasets with Persian patterns as well as Islamic patterns. Evaluation with cultural specialists confirms the feasibility of this approach, which received an average cultural fidelity rating of 4.6 ± 0.3 on a 5-point scale. The scores are significantly better than those typically associated with traditional generative methods. Commercial deployment analysis confirms effective portability across diverse hardware platforms, achieving 10 × model compression on mobile devices with acceptable quality degradation. This enables deployment across settings ranging from professional design studios and museum interactive displays to web-based cultural education platforms.

These contributions extend beyond technical advances, establishing systematic approaches for generating AI-driven content that is both culturally sensitive and contextually appropriate. This study demonstrates how culturally defined boundaries can simultaneously guide and constrain creative potential. At the field level, this work advances a replicable paradigm in which structured cultural knowledge, rather than purely statistical learning, governs generative AI behaviour, offering a methodological shift that may inform future research across digital humanities, computational creativity, and heritage informatics. The introduction of the Cultural Semantic Similarity metric provides the field with a quantifiable standard for evaluating cultural fidelity, addressing a long-standing absence of domain-specific evaluation criteria in heritage-oriented generative systems. This is relevant to future research on creative sites that deal with heritage. When diffusion models are applied with well-stratified cultural knowledge, it is possible to investigate means of understanding intangible cultural heritage. In this manner as well, it is possible for a usage that is cautious and honors traditional art as it is in contemporary society. Several directions remain for future work: automatic knowledge graph expansion from heritage archives, integration with text-guided generation for verbal cultural specification, and cross-modal extensions connecting patterns with traditional music. The framework has been tested with regional cultural institutions for exhibition displays, including collaborative projects with the Jilin Provincial Museum for interactive pattern presentations and preliminary trials with local textile enterprises exploring culturally-informed product design. These applications demonstrate commercial feasibility across museum installations, digital media production, and fashion design industries, contributing reproducible approaches for digitizing cultural elements while providing creative industries with tools for generating culturally informed design assets. This work contributes to the broader discourse on responsible AI deployment in cultural contexts, demonstrating that generative systems can operate within culturally-defined boundaries without compromising creative output quality. The methodology offers a replicable template for other heritage domains seeking to balance preservation imperatives with commercial innovation.

Author contributions

J.S. conceived the study, designed the framework architecture, implemented the diffusion model and knowledge graph integration, conducted all experiments, performed data analysis, and drafted the original manuscript. M.Y. supervised the research, acquired funding, provided conceptual guidance, and critically revised the manuscript for intellectual content. J.S. prepared Figs. 1–10 and Tables 1–5. All authors contributed to the interpretation of results and approved the final version of the manuscript.

Funding

This work was supported in part by the 2024 Social Science Research Project of the Education Department of Jilin Province, China under Grant JJKH20240113SK (Research on the Totem Art of Ethnic Minorities in Jilin Province), the Industry-sponsored Project of Beihua University under Grant 320125005 (Corporate Branding Design and Production), and the Beihua University Doctoral Research Start-up Project (IP Materialization Design of Jilin Folk Art in the New Cultural and Creative Context).

Data availability

The original contributions presented in the study are included in the article Material, further inquiries can be directed to the corresponding authors.

Declarations

Competing interests

The authors declare no competing interests.

Ethical approval

The study was approved by the Ethics Review Committee of Beihua University (Approval No. BHU-EC-2025-021; Approval Date: February 24, 2025). All methods were performed in accordance with relevant guidelines and regulations, including the Declaration of Helsinki. Written informed consent was obtained from all individual participants prior to their participation in the user evaluation.