VisTCP: A Visualization Framework to Construct Knowledge-Graph-Based Representation for Traditional Chinese Painting

Jul 16, 2026

Structured representation characterizes semantic objects and relationships in images, offering meaningful support for archaeology and art history research on Traditional Chinese Paintings (TCPs). Existing image-oriented structured representation approaches underperform on TCPs because TCP objects and events differ greatly from modern natural images and accurate identification remains difficult even for domain experts. VisTCP combines a TCP-oriented intelligent model with expert knowledge in a human-in-the-loop workflow, using semantic taxonomy building, expert-annotated training, joint embedding visualization for uncertainty, and iterative refinement validated through case studies and expert interviews.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Introduction

Motivation and challenges

Goal and proposed approach

Related work and knowledge graph construction

VisTCP: A Visualization Framework to Construct Knowledge-Graph-Based Representation for Traditional Chinese Painting Zhiguang Zhou, Fengling Zheng, Miaoxin Hu, Lina You, Jin Wen, Huan Liu, Wei Zhang, Dekun Qian, Yuhua Liu, Wei Chen, Yigang Wang*, and Yong Wang* arXiv :2607 .0584 1v 1 [ cs .HC] 7 Jul 2026 Abstract—Structured representation can characterize semantic objects and relationships in images. It provides a possible effective way for the semantic understanding of Traditional Chinese Paintings (TCPs) to better support archaeology and art history research. However, most image-oriented structured representation methods perform poorly on TCPs, due to two major challenges: 1) the objects and events of TCPs exhibit substantial differences from modern natural images, which results in semantic misunderstandings of TCPs; and 2) it is difficult to achieve accurate identification of ancient objects and events in TCPs, even for domain experts. In this paper, we propose VisTCP, a visualization framework that combines a TCPoriented intelligent model and expert knowledge, which enables art historians to achieve trustworthy structured representations of TCPs in a human-in-the-loop manner. Firstly, we conduct a pilot study with three domain experts to build a semantic taxonomy of TCPs. Then, expert-annotated data are used to train a TCP-oriented structured representation model, which can automatically extract meaningful objects and their relationships in TCPs. To inform users of the model uncertainty, we design a joint embedding visualization view to show the differences between expert annotations and model predictions. This allows users to refine the structured representation based on their domain knowledge, enabling iterative optimization of the model. Finally, we conduct a case study, a usage scenario, and expert interviews on a real dataset to demonstrate the effectiveness of VisTCP in supporting the structured representation and semantic understanding of TCPs. Index Terms—Traditional Chinese Paintings (TCPs), Visualization, Structured Representation, Image Understanding I. INTRODUCTION WITH the digitization of traditional Chinese paint ings(TCPs), computational methods have gained substantial attention in the fields of archaeology and art history, particularly for provenance analysis and authentication. However, art historians find that existing methods for structured Equal contribution: Fengling Zheng and Zhiguang Zhou contributed equally to this work. Zhiguang Zhou, Miaoxin Hu, Jin Wen, Dekun Qian, Yuhua Liu and Yigang Wang are with the School of Media and Design, Hangzhou Dianzi University (e-mail: [zhgzhou@hdu.edu.cn](zhgzhou@hdu.edu.cn), [miaoxinhu@outlook.com](miaoxinhu@outlook.com), [231330023@hdu.edu.cn](231330023@hdu.edu.cn), [qiandekun@hdu.edu.cn](qiandekun@hdu.edu.cn), [liuyuhua@hdu.edu.cn](liuyuhua@hdu.edu.cn), yi [gang.wang@hdu.edu.cn](gang.wang@hdu.edu.cn)). Fengling Zheng and Lina You are with the School of Computer Science, Hangzhou Dianzi University (e-mail: [fenglingzheng@hdu.edu.cn](fenglingzheng@hdu.edu.cn), [linayou@hdu.edu.cn](linayou@hdu.edu.cn)). Huan Liu, Wei Zhang and Wei Chen are with the State Key Lab of CAD&CG, Zhejiang University (email: [alisalh@zju.edu.cn](alisalh@zju.edu.cn), zw [yixian@zju.edu.cn](yixian@zju.edu.cn), [chenvis@zju.edu.cn](chenvis@zju.edu.cn)). Yong Wang is with the College of Computing and Data Science, Nanyang Technological University (e-mail: [yong-wang@ntu.edu.sg](yong-wang@ntu.edu.sg)). *Corresponding authors: Yigang Wang and Yong Wang understanding of TCPs, such as image classification, segmentation, and object detection, are inadequate for capturing the complex semantic content, which includes not only diverse objects, but also rich semantic events within the paintings. These methods often fail to support advanced semantic tasks such as semantic retrieval, question answering, and visual reasoning due to their inability to effectively represent seman

FAQs

What problem does VisTCP address for traditional Chinese paintings (TCPs)?

VisTCP targets the difficulty of producing trustworthy structured representations for TCPs, where existing methods struggle due to semantic mismatch with natural images and the challenge of accurately identifying ancient objects and events.

How does VisTCP incorporate expert knowledge with the model?

VisTCP uses a TCP-oriented structured representation model trained on expert-annotated data, and it enables a human-in-the-loop process where art historians can refine outputs using their domain knowledge.

How does VisTCP help users understand and correct model uncertainty?

VisTCP includes a joint embedding visualization view that highlights differences between expert annotations and model predictions, making uncertainty visible so users can iteratively improve the structured representation.

Jul 16, 2026

Upload by Theodore