Controllable Stable Diffusion Using ControlNet for Stylized Javanese Script Typography Synthesis
DOI:
https://doi.org/10.47709/brilliance.v6i4.9729Keywords:
Artificial Inteligence, Cultural Preservation, Diffusion Models, Image Synthesis, Natural Language Processing, Typographical GenerationAbstract
Preserving traditional indigenous scripts in digital environments requires robust computational frameworks that maintain strict orthographic fidelity alongside flexible artistic expression. Indigenous writing systems possess intricate non-linear structures, vertical character stacking, and delicate diacritics that frequently trigger severe structural distortion and character hallucinations in standard text-to-image generative architectures. This research aims to develop an automated hybrid pipeline that unites rule-based natural language processing with probabilistic generative artificial intelligence to synthesize high-fidelity stylized Javanese script calligraphy. The proposed method integrates linguistic normalization and deterministic transliteration to produce standardized visual guidance masks, followed by dual spatial conditioning that executes edge boundary extraction and depth estimation in parallel. Furthermore, low-rank adaptation modules are embedded into the latent diffusion process to extract complex visual material textures from curated paired synthetic datasets. Experimental evaluations demonstrate that the optimal parameter configuration successfully synthesizes five distinct aesthetic material styles with zero typographical errors and zero percent physical glyph deformation. The dual spatial constraints effectively lock vertical stacking components and microscopic diacritics while rendering realistic three-dimensional relief depth and specular surface lighting. In conclusion, the proposed hybrid architecture resolves structural disorientations in baseline diffusion models, providing a reliable technological infrastructure for digital cultural preservation, automated graphic typography production, and creative ethnic heritage dissemination.
References
Abdiansah, L., Sumarno, S., Eviyanti, A., & Azizah, N. L. (2025). Penerapan Algoritma Convolutional Neural Networks untuk Pengenalan Tulisan Tangan Aksara Jawa. MALCOM: Indonesian Journal of Machine Learning and Computer Science, 5(2), 496–504. https://doi.org/10.57152/malcom.v5i2.1814
Bosheah, Z., & Bilicki, V. (2025). Challenges in Generating Accurate Text in Images: A Benchmark for Text-to-Image Models on Specialized Content. Applied Sciences (Switzerland), 15(5). https://doi.org/10.3390/app15052274
Boudraa, M., Bennour, A., Mekhaznia, T., Alqarafi, A., Marie, R. R., Al-Sarem, M., & Dogra, A. (2024). Revolutionizing Historical Manuscript Analysis: A Deep Learning Approach with Intelligent Feature Extraction for Script Classification. Acta Informatica Pragensia, 13(2), 251–272. https://doi.org/10.18267/j.aip.239
Damayanti, F., Suzanti, I. O., Jauhari, A., & Herawati, S. (2024). Restoration of Javanese Characters Based on Wasserstein Generative Adversarial Network-Gradient Penalty. Mathematical Modelling of Engineering Problems, 11(12), 3447–3457. https://doi.org/10.18280/mmep.111223
Feng, Z., Chen, L., Sun, Y., Liu, J., & Feng, S. (2025). Simplifying Control Mechanism in Text-to-Image Diffusion Models. https://doi.org/10.1609/aaai.v38i4.32309
Hartley, Z. K. J., Lind, R. J., Pound, M. P., & French, A. P. (2024). Domain Targeted Synthetic Plant Style Transfer using Stable Diffusion, LoRA and ControlNet. https://doi.org/10.1109/CVPRW63382.2024.00546
Johny, C., Wolf¬sonkin, L., Gutkin, A., Roark, B., & Research, G. (2021). Finite¬state script normalization and processing utilities: The Nisaba Brahmic library. https://doi.org/10.18653/v1/2021.eacl-demos.3
Liu, Y., & Wang, H. (2025). Application of deep learning for transformation of Chinese traditional cultural narrative patterns and enhancement of cultural identity empowered by AIGC. Scientific Reports, 16(1). https://doi.org/10.1038/s41598-025-32302-5
Mahastama, A. W. (2022). Model Berbasis Aturan Untuk Transliterasi Bahasa Jawa Dengan Aksara Latin Ke Aksara Jawa. In Jurnal Buana Informatika (Vol. 13). https://doi.org/10.24002/jbi.v13i02.6526
Peong, K., Uchida, S., & Haraguchi, D. (2024). Typographic Text Generation with Off-the-Shelf Diffusion Model. Retrieved from http://arxiv.org/abs/2402.14314
Susanto, A., Mulyono, I. U. W., Sari, C. A., Rachmawanto, E. H., Setiadi, D. R. I. M., & Sarker, M. K. (2023). Handwritten Javanese script recognition method based 12-layers deep convolutional neural network and data augmentation. IAES International Journal of Artificial Intelligence, 12(3), 1448–1458. https://doi.org/10.11591/ijai.v12.i3.pp1448-1458
Tanveer, M., Wang, Y., Mahdavi-Amiri, A., & Zhang, H. (2023). DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion. https://doi.org/10.1109/ICCV51070.2023.00041
Wibowo, K. C., Damayanti, F., & Abdilqoyyim, F. (2025). Ancient Javanese Manuscript Reconstruction Using Generative Adversarial Network with StarGAN v2 Variations. Teknika, 14(1), 135–141. https://doi.org/10.34148/teknika.v14i1.1182
Yang, Z., Peng, D., Kong, Y., Zhang, Y., Yao, C., & Jin, L. (2023). FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning. https://doi.org/10.1609/aaai.v38i7.28482
Zhang, L., Rao, A., & Agrawala, M. (2023). Adding Conditional Control to Text-to-Image Diffusion Models. https://doi.org/10.1109/ICCV51070.2023.00355
Zhao, S., Chen, D., Chen, Y.-C., Bao, J., Hao, S., Yuan, L., & Wong, K.-Y. K. (2023). Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models. https://doi.org/10.48550/arXiv.2305.16322
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Zidan Amikul Afham, Muhammad Fachrie

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.















