AniFaceDrawing - Anime Portrait Exploration during Your Sketching

Aug 21, 2026

Upload by Kyle

The paper presents an AI-assisted system that converts a sequence of incomplete freehand sketches into high-quality anime portraits as the user draws stroke by stroke. Maintaining quality from low-completion sketches remains difficult for conditional image generation, especially for anime styles that are more abstract than realistic faces. The method applies latent space exploration of StyleGAN with two-stage training and stroke-level disentanglement, aligning sketch attributes with disentangled latent structural codes. Qualitative and quantitative evaluations, plus a user study, confirm effectiveness for art creation assistance in the anime style.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Introduction

Problem Setup

Method Overview

Evaluation

AniFaceDrawing: Anime Portrait Exploration during Your Sketching ZHENGYU HUANG, Japan Advanced Institute of Science and Technology, Japan HAORAN XIE, Japan Advanced Institute of Science and Technology, Japan TSUKASA FUKUSATO, Waseda University, Japan KAZUNORI MIYATA, Japan Advanced Institute of Science and Technology, Japan arXiv :2306 .07476v1 [ cs .CV] 13 Jun 2023 In this paper, we focus on how artificial intelligence (AI) can be used to assist users in the creation of anime portraits, that is, converting rough sketches into anime portraits during their sketching process. The input is a sequence of incomplete freehand sketches that are gradually refined stroke by stroke, while the output is a sequence of high-quality anime portraits that correspond to the input sketches as guidance. Although recent GANscan generate high quality images, it is a challenging problem to maintain the high quality of generated images from sketches with a low degree of completion due to ill-posed problems in conditional image generation. Even with the latest sketch-to-image (S2I) technology, it is still difficult to create high-quality images from incomplete rough sketches for anime portraits since anime style tend to be more abstract than in realistic style. To address this issue, we adopt a latent space exploration of StyleGAN with a two-stage training strategy. We consider the input strokes of a freehand sketch to correspond to edge information-related attributes in the latent structural code of StyleGAN, and term the matching between strokes and these attributes “stroke-level disentanglement.” In the first stage, we trained an image encoder with the pre-trained StyleGAN model as a teacher encoder. In the second stage, we simulated the drawing process of the generated images without any additional data (labels) and trained the sketch encoder for incomplete progressive sketches to generate high-quality portrait images with feature alignment to the disentangled representations in the teacher encoder. We verified the proposed progressive S2I system with both qualitative and quantitative evaluations and achieved high-quality anime portraits from incomplete progressive sketches. Our user study proved its effectiveness in art creation assistance for the anime style. CCS Concepts: • Computing methodologies → Graphics systems and interfaces; Image manipulation; Machine learning. Additional KeyWords and Phrases: Stroke-level Disentanglement, StyleGAN, Anime Portrait, Disentanglement Learning, Freehand Sketching ACM Reference Format: Zhengyu Huang, Haoran Xie, Tsukasa Fukusato, and Kazunori Miyata. 2023. AniFaceDrawing: Anime Portrait Exploration during Your Sketching . 1, 1 (June 2023), 11 pages. [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1145/nnnnnnn.nnnnnnn) 1 INTRODUCTION Using AI to assist the general user in creating a professional anime portrait is not a trivial task. As a popular drawing style, the anime Authors’ addresses: Zhengyu Huang, Japan Advanced Institute of Science and Technology, Nomi, Japan, [huang.zhengyu@jaist.ac.jp](huang.zhengyu@jaist.ac.jp); Haoran Xie, Japan Advanced Institute of Science and Technology, Nomi, Japan, [xie@jaist.ac.jp](xie@jaist.ac.jp); Tsukasa Fukusato, Waseda University, Tokyo, Japan, [tsukasafukusato@waseda.jp](tsukasafukusato@waseda.jp); Kazunori Miyata, Japan Advanced Institute of Science and Technology, Nomi, Japan, [miyata@jaist.ac.jp](miyata@jaist.ac.jp). Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s) . © 2023 Copyright held by the owner/author(s) . XXXX-XXXX/2023/6-ART [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https:

FAQs

How does the proposed system generate anime portraits from user sketches?

It takes a sequence of progressively refined freehand sketches as input and produces a corresponding sequence of high-quality anime portraits to guide the user during sketching.

Why is sketch-to-anime portrait generation difficult with incomplete sketches?

Incomplete, low-detail strokes create ill-posed conditional image generation, and anime styling is more abstract than realistic facial rendering, so quality degrades as sketches are less complete.

What is “stroke-level disentanglement” in the method?

It matches input sketch strokes to edge information-related attributes in StyleGAN’s latent structural code, enabling progressive feature alignment during generation.

Aug 21, 2026

Upload by Kyle