A Vision Based System for Guided and Collaborative Reconstruction of Fragmented Documents

Jul 16, 2026

Upload by Ivy

The paper develops and evaluates a collaborative, real-time reconstruction system for fragmented paper documents used in cultural heritage preservation. A collaborative robot (cobot) positions fragments with a purpose-built vacuum-based suction attachment that protects fragile material while enabling gentle, precise handling. The setup achieves 0.57 mm positioning repeatability for 8 cm² fragments and supports both visual-guided manual placement and fully automated cobot placement. AI-driven image interpretation is assessed for segmentation and positioning, and SE2-LoFTR is selected for robust, detector-free feature matching under rotation, scale changes, and fragment damage.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

Motivation

System overview and hardware

Contributions

Contribution 1

Contribution 2

A Vision Based System for Guided and Collaborative Reconstruction of Fragmented Documents Oliver Krumpek and Diana Leo Fraunhofer IPK, dept. Machine Vision, Pascalstr. 8-9, 10587 Berlin, Germany arXiv :2607 .0362 1v 1 [ cs .CV] 3 Jul 2026 Abstract. This paper presents the development and evaluation of a collaborative system for real-time reconstruction of fragmented paper documents in the context of cultural heritage preservation. The developed system includes a collaborative robot, or cobot, that can fully manage the positioning of paper fragments using a specially designed vacuum-based suction attachment. This attachment enables gentle and precise positioning, ensuring the preservation of fragile materials. With this device, we are able to achieve a positioning repeatability of 0 .57mm for fragments of 8cm² . The system offers users the flexibility to choose between manual positioning, with visual guidance, or fully automated positioning performed by the cobot. To further improve the reconstruction process, AI methods for image interpretation, specifically for segmentation and positioning tasks, were applied and evaluated for their applicability to template-based reconstruction of damaged paper fragments. Our investigation provides critical insights into the performance of different local feature matching methods under different document types, taking into account rotation, scale robustness, and the degree of damage to the fragments. With a focus on the reconstruction of damaged and optically altered archival material, SE2-LoFTR, a detector-free local feature matching method, was chosen as the preferred method for the system due to its robust performance in our experiments. Keywords: Automatic Reconstruction · Assistance System · Fragmented Paper · Feature Matching · Human Machine Interaction · Collaborative Systems 1 Motivation The Fraunhofer Institute for Production Systems and Design Technology (IPK) is part of the Research Alliance for Cultural Heritage (FalKe), which comprises the Fraunhofer-Gesellschaft, the Leibniz Association and the Prussian Cultural Heritage Foundation [1] . We claim that the use of automation technology, including collaborative robots (cobots), has the potential to contribute to the feasibility of reconstruction projects. This is particularly true for projects with high repeatability of the handling tasks. Using advanced sensors and vision systems, cobots can carefully navigate and manipulate delicate materials, taking over the task of positioning fragments of different types and sizes. The integration of computer vision and artificial intelligence for automatic image analysis further enhances the capabilities for accurate reconstruction. The use of automation technology streamlines and accelerates the reconstruction process, resulting in improved efficiency. Fig. 1: Annotated image of the overall hardware setup, illustrating the key components. The system includes an industrial camera that digitises the fragments, and a projector that displays the user interface, providing realtime feedback during the reconstruction process. The user can either manually place the fragment in the found target position or delegate this task to the centrally positioned cobot, which handles the precise placement upon request. Assistive systems that combine human expertise with machine capabilities allow experts to focus on more complex aspects of the reconstruction process such as process control, correction and validation. This human-machine synergy improves the overall reconstruction process and contributes to the preservation of cultural artefacts. Furthermore, the application of collaborative robotics and automation technology extends beyond the reconstruction of paper documents. The principles and methodologies developed in this context can be adapted to the restoration of works of art, the reconstruction of archaeological artefacts and the preservation of historical manuscripts. The system

FAQs

What is the main goal of the proposed system?

To enable real-time, collaborative reconstruction of fragmented paper documents for cultural heritage preservation, combining robot automation with human control when needed.

How does the cobot position fragile paper fragments safely?

It uses a material-protective, vacuum-based suction attachment with sensors to gently and precisely place fragments while minimizing physical impact.

Which AI-based feature matching method is selected and why?

SE2-LoFTR is chosen because it delivers robust performance for damaged and optically altered archival materials, particularly under rotation, scale robustness, and varying damage levels.

Jul 16, 2026

Upload by Ivy