MF-Net

"Hmm, can Generative AI help us design fonts?”

[GitHub Repo] [DOI: 10.1145/3503161.3548414]

MF-Net: A Novel Few-shot Stylized Multilingual Font Generation Method

• Paper presented at the 30th ACM International Conference on Multimedia (MM ‘22)
• Authors: Yufan Zhang, Junkai Man, Peng Sun*

Video Presentation

---

Introduction

Designing fonts is never a simple task, especially when it comes to non-Latin languages like Chinese, which have intricate glyph patterns and a high number of characters. Furthermore, in the real world, creating a multilingual font poses an even greater challenge, requiring years of research, expertise in using numerous professional tools, and collaboration among multiple font designers specializing in various languages. However, a multilingual typeface is indispensable for many applications, including multilingual websites, documents, and books. Therefore, it is crucial to develop a system that can automatically generate stylized multilingual fonts.

In this project, I, together with my roommate, Junkai, and Prof. Peng Sun attempted to address this issue using deep learning techniques. Specifically, our study aimed to develop a highly effective few-shot multilingual stylized font generator. This generator would be capable of producing an entire font family in all unseen languages that share the desired font style, given only a limited number of font images representing the target style from one language. Our work shows promise as it significantly reduces the cost associated with multilingual stylistic font design, enabling anyone to create their own fonts without years of extensive training.

Methodology

Developing such an intelligent system faces three significant obstacles:

1. Few-shot learning: The system must learn the font style from a small number of reference samples. This means it needs to quickly grasp the characteristics of the desired style and generate accurate results with limited training data.
2. Font style transfer between multiple languages: The system should be capable of transferring font styles across different languages, even for languages that it hasn't been explicitly trained on. This requires the ability to capture the essence of the style and apply it appropriately to various linguistic systems.
3. Output generation by direct inference: Instead of relying on model fine-tuning, the system aims to generate the desired output directly through inference. This approach reduces the need for extensive fine-tuning and allows for more efficient font generation.

Overview of the network structure of MF-Net

We introduced an end-to-end model based on a generative adversarial network (GAN) to tackle these challenges. Specifically, the model incorporates two separate encoders within the generator to separate the content and style information of each font image. Taking inspiration from a paper published in ICCV '21, I proposed a unique skip connection called the language complexity-aware skip connection for our cross-language font generation task, as both studies aim to achieve style migration across multiple style domains. This innovative skip connection allows for adaptive preservation of the structural information of the content. Additionally, I made modifications to the loss function to ensure the decoupling of information by enforcing the distinct roles of the two encoders.

Results Gallery

Results of MF-Net on seen languages

Results of MF-Net on unseen languages

Cite Us

@inproceedings{10.1145/3503161.3548414,
   author = {Zhang, Yufan and Man, Junkai and Sun, Peng},
   title = {MF-Net: A Novel Few-Shot Stylized Multilingual Font Generation Method},
   year = {2022},
   isbn = {9781450392037},
   publisher = {Association for Computing Machinery},
   address = {New York, NY, USA},
   url = {https://doi.org/10.1145/3503161.3548414},
   doi = {10.1145/3503161.3548414},
   booktitle = {Proceedings of the 30th ACM International Conference on Multimedia},
   pages = {2088–2096},
   numpages = {9},
   keywords = {font design, image synthesis, few-shot learning, style transfer},
   location = {Lisboa, Portugal},
   series = {MM '22}
}