When Simpler Is Better - Evaluating Translation Pipelines for Medieval Latin Manuscripts - nguyen Kim Hai Bui et al.

Sep 21, 2026

Uploaded by Asher

Despite rapid progress in machine translation, vision language models face major obstacles when translating historical documents. Medieval Latin manuscripts intensify core NLP challenges: low-resource transliteration, archaic vocabulary, and noisy signals caused by scribal shorthand, ligatures, and parchment degradation. The work proposes a systematic, end-to-end evaluation framework for the full image-to-translation pipeline in this setting and benchmarks on the CATMuS Latin dataset.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Abstract

Introduction

When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts Nguyen Kim Hai Bui 1 ∗ Md. Easin Arafat 1 ∗ Tamás Gábor Orosz 1 Mufti Mahmud2 1Eötvös Loránd University 2 King Fahd University of Petroleum and Minerals {qmibhu,arafatmdeasin}@inf.elte .hu arXiv :2607 .03836v 1 [ cs .CV] 4 Jul 2026 Abstract Despite remarkable progress in machine translation, Vision Language Models (VLMs) struggle on historical manuscripts, a domain that stresses core Natural Language Processing (NLP) capabilities: low-resource transliteration, archaic vocabulary, and noisy input signals. We present a systematic framework for evaluating the full image-to-translation pipeline on medieval Latin manuscripts, a setting in which scribal shorthand, ligatures, and parchment degradation expose failure modes that are invisible in clean-text benchmarks. Benchmarking on the CATMuS Latin dataset reveals a specialization gap: domain-specific Optical Character Recognition (OCR) models reduce character error rate by up to 4.3 × compared to general-purpose VLMs, despite operating at orders of magnitude fewer parameters. We introduce the Interpres-ParallelCorpus (IPC), a novel dataset comprising 1,383 aligned manuscript image lines, transcriptions, and expert translations, the first of its kind for medieval Latin. Our experiments uncover a complexity paradox: the simplest pipeline, a specialized OCR model feeding directly into a VLM, outperforms all multi-component variants. Adding retrievalaugmented generation (RAG) or post-OCR correction introduces prompt saturation and error propagation that degrade aggregate translation quality. These findings offer both a new benchmark and practical guidance for deploying translation systems in low-resource historical settings. 1 Introduction Medieval Latin manuscripts are the foundational primary sources for European history, philosophy, and law from late antiquity to the early modern period (Bischoff, 1990) . From royal charters to philosophical treatises, these documents contain the blueprint of Western intellectual heritage. ∗Equal contribution Character Error Rate ( ) 1.0 0.8 0.6 0.4 0.2 | | | | | | | --- | --- | --- | --- | --- | | | | |

| | | | | |
| | | | | |
| | | better
| | | | | | | | | | | 1 01 1 02 1 03 1 04 Domain-tuned TrOCR General-purpose VLM Traditional OCR Parameter Size (Millions, log scale) Figure 1: Scatter plot demonstrating the specialization gap. Small, domain-tuned TrOCR models achieve the lowest CER, outperforming general-purpose VLMs that are larger in parameter size. Mistral-OCR-3 is assumed to be the largest, as the creator doesn’t publish its actual size. However, the vast majority of these collections remain untranscribed and untranslated, locked behind a paleographic barrier that requires years of specialized training to overcome. Historical text is not merely a humanities concern – it serves asa stress test for NLP robustness. Archaic orthographic conventions, low-resource scribal abbreviations, and morphological variation that deviates sharply from modern token distributions probe fundamental limitations in how current VLMs handle distributional shift. While some domains in machine translation have reached near-human parity (Hassan et al., 2018), historical manuscripts present a unique frontier defined by scribal abbreviations, non-standardized ligatures, and physical parchment degradation (Derolez, 2003) . The typical approach to unlocking these texts involves a multi-stage process: OCR, post-OCR correction, and semantic translation. Recent advances in Transformer-based OCR (TrOCR) (Liet al., 2023) and character-level language model- ing (ByT5) (Xue et al., 2022) have provided powerful building blocks for this task. However, thereis a lack of systematic evaluation regarding how these components should be integrated into an endto-end (E2E) pipeline. Specifically, it remains unclea

FAQs

Why are medieval Latin manuscripts difficult for vision language translation systems?

They introduce low-resource transliteration, archaic vocabulary, and noisy inputs from scribal shorthand, ligatures, and parchment degradation, which expose failure modes not present in clean-text benchmarks.

What does the CATMuS Latin benchmarking show about specialization?

Domain-specific OCR models reduce character error rate by up to 4.3× compared with general-purpose VLMs, even when using far fewer parameters.

What is the key conclusion about pipeline complexity?

The simplest pipeline—specialized OCR feeding directly into a VLM—outperforms more complex multi-component variants, because added RAG or post-OCR correction can introduce prompt saturation and error propagation.

Sep 21, 2026

Uploaded by Asher