IMPACT: A Large-scale Integrated Multimodal Patent Analysis and CreaTion Dataset for Design Patents - Paper Datasets and Benchmarks Track

Oct 24, 2026

IMPACT introduces a large-scale integrated multimodal patent dataset focused on design patents, pairing detailed captions with design patent figures for multimodal learning. The dataset includes half a million design patents with 3.61 million figures and accompanying captions drawn from USPTO-granted patents across 2007–2022. By adding richer, viewpoint-consistent descriptions, it enables classification and retrieval tasks that existing design patent resources lack. Results show caption integration improves model performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages27

Reading Time1 h 8 min

AccessFree to download

Introduction

Existing works on patent datasets

Limitations of existing datasets

Our contributions

IMPACT dataset overview

IMPACT: A Large-scale Integrated Multimodal Patent Analysis and Creation Dataset for Design Patents Homaira Huda Shomee Zhu Wang Sourav Medya Sathya N. Ravi Department of Computer Science, University of Illinois Chicago {hshome2,zwang260,medya,[sathya}@uic.edu](sathya}@uic.edu) Abstract In this paper, we introduce IMPACT (Integrated Multimodal Patent Analysis and CreaTion Dataset for Design Patents), a large-scale multimodal patent dataset with detailed captions for design patent figures. Our dataset includes half a million design patents comprising 3.61 million figures along with captions from patents granted by the United States Patent and Trademark Office (USPTO) over a 16-year period from 2007 to 2022 . We incorporate the metadata of each patent application with elaborate captions that are coherent with multiple viewpoints of designs. Even though patents themselves contain a variety of design figures, titles, and descriptions of viewpoints, we find that they lack detailed descriptions that are necessary to perform multimodal tasks such as classification and retrieval. IMPACT closes this gap thereby providing researchers with necessary ingredients to instantiate a variety of multimodal tasks. Our dataset has a huge potential for novel design inspiration and can be used with advanced computer vision models in tandem. We perform preliminary evaluations on the dataset on the popular patent analysis tasks such as classification and retrieval. Our results indicate that integrating images with generated captions significantly improves the performance of different models on the corresponding tasks. Given that design patents offer various benefits for modeling novel tasks, we propose two standard computer vision tasks that have not been investigated in analyzing patents as future directions using IMPACT as a benchmark viz., 3D Image Construction and Visual Question Answering (VQA) . To facilitate research in these directions, we make our IMPACT dataset and the code/models used in this work publicly available here. 1 Introduction Design patents are important for intellectual property protection in industries where aesthetics and user experience are predominant, such as consumer electronics, fashion, and automotive design [27, 1] . Unlike utility patents, which protect the functional aspects of an invention, design patents safeguard the ornamental or aesthetic features of a product. This includes elements such as shape, surface ornamentation, and overall visual appearance. Design patents provide inventors with exclusive rights to their creations and prevent others from producing, selling, or using designs that are substantially similar. In the past few years, the number of design patents granted in the US is increasing. Though these are publicly available by law, the data is not well-organized for patent analyses which could help patent offices as well as can serve as a benchmark for the machine learning community. Existing works on patent datasets. A variety of patent data formats, including XML, TSV, TIFF, and PDF, are publicly accessible for bulk download from multiple sources. One notable dataset, USPTO2M [33], comprises approximately 2 million utility patents specifically curated for classification tasks. Another dataset, BIGPATENT [48], contains a corpus of 1.3 million utility patents, exclusively in textual format. Additionally, the Harvard University Patent Dataset (HUPD) [55] 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks. Figure 1: The main components of IMPACT dataset. We collect and pre-process the raw data to construct a comprehensive well-structure and accessible design patent dataset. We integrate multimodal information and design various training and application scenarios. consists of a broader range of data fields, compiling 4.5 million patents intended for multipurpose patent analysis. These datasets focus only on utility patents. Despit

FAQs

What is IMPACT and what does it provide for design patents?

IMPACT is a large-scale multimodal dataset that links design patent figures with detailed captions, enabling multimodal analysis. It targets tasks that require richer figure descriptions beyond existing patent text.

How large is the IMPACT dataset and what is its coverage period?

IMPACT includes about half a million design patents and 3.61 million figures, with captions from USPTO-granted patents. The coverage spans 2007 to 2022 over a 16-year period.

Why do the authors say existing datasets are insufficient for multimodal patent tasks?

Existing work often uses single-modality data (only text or only images) and lacks organized design-patent resources. For design patents specifically, descriptive figure captions are missing, limiting classification, retrieval, and prior-art style searches.

Oct 24, 2026

Upload by River Wang