UniGlyph - Unified Segmentation-Conditioned Diffusion for Precise Visual Text

Aug 29, 2026

Uploaded by Theodore

Text-to-image generation has progressed rapidly, yet producing accurate visual text remains difficult because glyphs blur, semantics drift, and typographic style control is limited. Existing approaches rely on prerendered glyph images as conditions, which lose critical font and color information and force complex multi-branch architectures. UniGlyph introduces segmentation-guided unified conditioning using pixel-level visual text masks, plus a fine-tuned bilingual segmentation model and an adaptive diffusion framework with region-specific loss. Results on AnyText and new benchmarks (GlyphMM and MiniText) show strong gains, especially for small and complex layouts.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Abstract

Introduction

Problem: blurred glyphs and weak style control

Proposed method: UniGlyph

Training data and benchmarks

arXiv :2507 .00992v2 [ cs .CV] 2 Jul 2025 UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis Yuanrui Wang 1 ,2 ,∗ , Cong Han2 ,∗ , Yafei Li2 , Zhipeng Jin2 , Xiawei Li2 , SiNan Du 1 , Wen Tao2 , Shuanglong Li2 , Yi Yang2 ,†, Chun Yuan 1 ,†, Liu Lin2 1Tsinghua University 2Baidu Inc. [wang-yr23@mails.tsinghua.edu.cn](wang-yr23@mails.tsinghua.edu.cn) , [hancong01@baidu.com](hancong01@baidu.com) , [liyafei03@baidu.com](liyafei03@baidu.com) [dsn23@mails.tsinghua.edu.cn](dsn23@mails.tsinghua.edu.cn) , [jinzhipeng@baidu.com](jinzhipeng@baidu.com) , [lixiawei01@baidu.com](lixiawei01@baidu.com) , [taowen02@baidu.com](taowen02@baidu.com) [lishuanglong@baidu.com](lishuanglong@baidu.com) , [yangyi15@baidu.com](yangyi15@baidu.com) , [liulin03@baidu.com](liulin03@baidu.com) Abstract Text-to-image generation has greatly advanced content creation, yet accurately rendering visual text remains a key challenge due to blurred glyphs, semantic drift, and limited style control. Existing methods often rely on prerendered glyph images as conditions, but these struggle to retain original font styles and color cues, necessitating complex multi-branch designs that increase model overhead and reduce flexibility. To address these issues, we propose a segmentation-guided framework that uses pixel-level visual text masks—rich in glyph shape, color, and spatial detail—as unified conditional inputs. Our method introduces two core components: (1) a fine-tuned bilingual segmentation model for precise text mask extraction, and (2) a streamlined diffusion model augmented with adaptive glyph conditioning and a region-specific loss to preserve textual fidelity in both content and style. Our approach achieves state-of-the-art performance on the AnyText-benchmark, significantly surpassing prior methods in both Chinese and English settings. To enable more rigorous evaluation, we also introduce two new benchmarks: GlyphMM-benchmark for testing layout and glyph consistency in complex typesetting, and MiniText-benchmark for assessing generation quality in small-scale text regions. Experimental results show that our model outperforms existing methods by a large margin in both scenarios, particularly excelling at small text rendering and complex layout preservation, validating its strong generalization and deployment readiness. ∗ Equal Contribution.† Corresponding Authors. 1. Introduction In recent years, deep learning has achieved impressive success in various domains of machine learning and artificial intelligence, including but not limited to computer vision, image generation, image processing, and controllable synthesis [1, 5–7, 10, 25–28, 30, 35, 41, 42, 44, 45, 54, 55] . However, generating precise visual text (glyphs) remains an unsolved challenge, with persistent issues including blurred character edges, semantic inconsistencies, and poor control over typographic attributes like font styles and colors. Current solutions addressing this problem predominantly adopt three paradigms: (1) inpainting-based methods (e.g., Text-Diffuser [3]), which struggle with text-background harmonization; (2) embedding-injection approaches (e.g., Glyph-ByT5-v2 [21]), limited by predefined typographic constraints; and (3) ControlNet-based architectures, exemplified by state-of-the-art methods AnyText [38] and GlyphDraw2 [23], which currently set the performance benchmark but suffer from fundamental limitations. The ControlNet-based paradigm, while effective, relies on pre-rendered glyph images that inherently discard critical typographic information. Specifically, these rendered conditions preserve only glyph shapes and positions while losing original font characteristics and color details. This incomplete signal forces models to learn implicit mappings between synthetic glyphs (typically in default fonts) and real-world typographic variations during training. To compensate, existing methods introduce auxiliary modules for posi

FAQs

Why do existing visual-text generation methods struggle with font and color fidelity?

They often condition diffusion models on prerendered glyph images, which preserve only shape and position while discarding original font characteristics and color cues. This degraded conditioning forces models to learn implicit mappings and leads to weaker control.

What is UniGlyph’s core idea for generating precise visual text?

UniGlyph replaces rendered glyph-condition signals with pixel-accurate visual text segmentation masks. These masks preserve glyph attributes such as shape, spatial layout, font style, and color at full resolution.

How does UniGlyph maintain textual fidelity during diffusion generation?

It combines an adaptive glyph conditioning mechanism with a region-specific loss designed to preserve glyph correctness in both content and style.

Aug 29, 2026

Uploaded by Theodore