Controllable Stable Diffusion Using ControlNet for Stylized Javanese Script Typography Synthesis

Authors

  • Zidan Amikul Afham Department of Informatics, Universitas Teknologi Yogyakarta, Indonesia
  • Muhammad Fachrie Universitas Teknologi Yogyakarta, Indonesia

DOI:

https://doi.org/10.47709/brilliance.v6i4.9729

Keywords:

Artificial Inteligence, Cultural Preservation, Diffusion Models, Image Synthesis, Natural Language Processing, Typographical Generation

Abstract

Preserving traditional indigenous scripts in digital environments requires robust computational frameworks that maintain strict orthographic fidelity alongside flexible artistic expression. Indigenous writing systems possess intricate non-linear structures, vertical character stacking, and delicate diacritics that frequently trigger severe structural distortion and character hallucinations in standard text-to-image generative architectures. This research aims to develop an automated hybrid pipeline that unites rule-based natural language processing with probabilistic generative artificial intelligence to synthesize high-fidelity stylized Javanese script calligraphy. The proposed method integrates linguistic normalization and deterministic transliteration to produce standardized visual guidance masks, followed by dual spatial conditioning that executes edge boundary extraction and depth estimation in parallel. Furthermore, low-rank adaptation modules are embedded into the latent diffusion process to extract complex visual material textures from curated paired synthetic datasets. Experimental evaluations demonstrate that the optimal parameter configuration successfully synthesizes five distinct aesthetic material styles with zero typographical errors and zero percent physical glyph deformation. The dual spatial constraints effectively lock vertical stacking components and microscopic diacritics while rendering realistic three-dimensional relief depth and specular surface lighting. In conclusion, the proposed hybrid architecture resolves structural disorientations in baseline diffusion models, providing a reliable technological infrastructure for digital cultural preservation, automated graphic typography production, and creative ethnic heritage dissemination.

References

Abdiansah, L., Sumarno, S., Eviyanti, A., & Azizah, N. L. (2025). Penerapan Algoritma Convolutional Neural Networks untuk Pengenalan Tulisan Tangan Aksara Jawa. MALCOM: Indonesian Journal of Machine Learning and Computer Science, 5(2), 496–504. https://doi.org/10.57152/malcom.v5i2.1814

Bosheah, Z., & Bilicki, V. (2025). Challenges in Generating Accurate Text in Images: A Benchmark for Text-to-Image Models on Specialized Content. Applied Sciences (Switzerland), 15(5). https://doi.org/10.3390/app15052274

Boudraa, M., Bennour, A., Mekhaznia, T., Alqarafi, A., Marie, R. R., Al-Sarem, M., & Dogra, A. (2024). Revolutionizing Historical Manuscript Analysis: A Deep Learning Approach with Intelligent Feature Extraction for Script Classification. Acta Informatica Pragensia, 13(2), 251–272. https://doi.org/10.18267/j.aip.239

Damayanti, F., Suzanti, I. O., Jauhari, A., & Herawati, S. (2024). Restoration of Javanese Characters Based on Wasserstein Generative Adversarial Network-Gradient Penalty. Mathematical Modelling of Engineering Problems, 11(12), 3447–3457. https://doi.org/10.18280/mmep.111223

Feng, Z., Chen, L., Sun, Y., Liu, J., & Feng, S. (2025). Simplifying Control Mechanism in Text-to-Image Diffusion Models. https://doi.org/10.1609/aaai.v38i4.32309

Hartley, Z. K. J., Lind, R. J., Pound, M. P., & French, A. P. (2024). Domain Targeted Synthetic Plant Style Transfer using Stable Diffusion, LoRA and ControlNet. https://doi.org/10.1109/CVPRW63382.2024.00546

Johny, C., Wolf¬sonkin, L., Gutkin, A., Roark, B., & Research, G. (2021). Finite¬state script normalization and processing utilities: The Nisaba Brahmic library. https://doi.org/10.18653/v1/2021.eacl-demos.3

Liu, Y., & Wang, H. (2025). Application of deep learning for transformation of Chinese traditional cultural narrative patterns and enhancement of cultural identity empowered by AIGC. Scientific Reports, 16(1). https://doi.org/10.1038/s41598-025-32302-5

Mahastama, A. W. (2022). Model Berbasis Aturan Untuk Transliterasi Bahasa Jawa Dengan Aksara Latin Ke Aksara Jawa. In Jurnal Buana Informatika (Vol. 13). https://doi.org/10.24002/jbi.v13i02.6526

Peong, K., Uchida, S., & Haraguchi, D. (2024). Typographic Text Generation with Off-the-Shelf Diffusion Model. Retrieved from http://arxiv.org/abs/2402.14314

Susanto, A., Mulyono, I. U. W., Sari, C. A., Rachmawanto, E. H., Setiadi, D. R. I. M., & Sarker, M. K. (2023). Handwritten Javanese script recognition method based 12-layers deep convolutional neural network and data augmentation. IAES International Journal of Artificial Intelligence, 12(3), 1448–1458. https://doi.org/10.11591/ijai.v12.i3.pp1448-1458

Tanveer, M., Wang, Y., Mahdavi-Amiri, A., & Zhang, H. (2023). DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion. https://doi.org/10.1109/ICCV51070.2023.00041

Wibowo, K. C., Damayanti, F., & Abdilqoyyim, F. (2025). Ancient Javanese Manuscript Reconstruction Using Generative Adversarial Network with StarGAN v2 Variations. Teknika, 14(1), 135–141. https://doi.org/10.34148/teknika.v14i1.1182

Yang, Z., Peng, D., Kong, Y., Zhang, Y., Yao, C., & Jin, L. (2023). FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning. https://doi.org/10.1609/aaai.v38i7.28482

Zhang, L., Rao, A., & Agrawala, M. (2023). Adding Conditional Control to Text-to-Image Diffusion Models. https://doi.org/10.1109/ICCV51070.2023.00355

Zhao, S., Chen, D., Chen, Y.-C., Bao, J., Hao, S., Yuan, L., & Wong, K.-Y. K. (2023). Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models. https://doi.org/10.48550/arXiv.2305.16322

Downloads

Published

2026-11-01

How to Cite

Afham, Z. A., & Fachrie, M. (2026). Controllable Stable Diffusion Using ControlNet for Stylized Javanese Script Typography Synthesis. Brilliance: Research of Artificial Intelligence, 6(4), 723–732. https://doi.org/10.47709/brilliance.v6i4.9729

Citation Tracker

Citation data is temporarily unavailable.

Source: OpenAlex