From Panels to Prose - Generating Literary Narratives from Comics - Supplementary Material - A Distinctive contributions of the Magiv3 model

Aug 26, 2026

Supplementary material details the Magiv3 model’s distinctive design for generating literary narratives from comics. It contrasts Magiv3 with Florence-2 and earlier Magiv variants, highlighting improved character grounding, clustering, and speaker diarisation via a unified architecture and next-token prediction behavior that avoids threshold tuning. The document further expands a training-free, zero-shot approach that pre-grounds character captions using detected panels and visual prompting, then discusses limitations and cross-lingual generalisation for Western comics and non-English text.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

A. Distinctive contributions of the Magiv3 model

Comparison with prior works (Table 5)

Robustness to non-standard panel layouts

Generalisation across Western comics and other languages

B. More on character grounding

Training-free zero-shot pre-grounded captions

Visual prompting with bounding-box IDs

GPT-4o-based caption formatting

From Panels to Prose: Generating Literary Narratives from Comics Supplementary Material A. Distinctive contributions of the Magiv3 model While the main contribution of this work is the pipeline we propose as a whole, along with the dataset annotations which permit future research and benchmarking, in this section we want to draw attention to the proposed Magiv3 model. Since this model is built on top of Florence-2 [43], we want to be clear and transparent about how our model is distinctive from prior works. Tab. 5 provides an overview. Detection Character Clustering Text-Character Matching OCR Captioning Character Grounding DASS [36] Magiv1 [30] Magiv2 [31] Florence-2 [43] Magiv3 [Ours] Characters Characters, Texts, Panels Characters, Texts, Panels, Tails Zero-shot, performs poorly Characters, Texts, Panels, Tails No Yes, using a separate character crop encoder Yes, using a separate character crop encoder No Yes No Yes Yes No Yes No No, uses a separate model No, uses a separate model Yes, but line-by-line instead of text-blocks Yes No No No Yes, low quality No No No No Not really Yes Table 5 . Comparison of Magiv3 with prior works. The notable advantage of Magiv3 is that it is a uniﬁed architecture for many manga-related tasks with a single set of model weights. Despite being built on top of Florence-2, Magiv3’s abilities are signiﬁcantly different. For instance, as shown in Fig. 7, Florence-2 cannot meaningfully ground characters in the given captions. It has no notion of context speciﬁc phrase grounding and ultimately ﬁnds “class labels” from the captions and predicts boxes for them. Florence-2 also cannot perform character clustering or speaker diarisation which Magiv3 can. Conversely, Magiv3 has not been trained to generate captions due to the high training costs required, while Florence-2 can generate [low quality] captions. Additionally, Magiv3 can do everything Magiv1, Magiv2 etc. can (and more), without needing a separate crop encoder trained contrastively, or a separate OCR model, or post-processing for ordering. In fact, due to the next token prediction paradigm, we do not have to tweak any thresholds to ﬁlter out low conﬁdence predictions; unlike prior works which use ﬁnicky hyperparameters which are sensitive to near-boundary predictions. As shown in Fig. 8, the optimal threshold for character detection in Magiv2 tends to either miss predictions or add false positives. In contrast, Magiv3 works much better and hence has a much higher precision for character detection (as noted in the main paper) . The model is able to handle non-standard grid layouts. For example, in Fig. 8 there is a “non standard” case with a borderless panel and the layout isn’t a typical grid. In general, the model is extremely robust to various panel arrangements. We include a few more examples in Fig. 9. Finally, in terms of generalisation, even though we exclusively train Magiv3 on Japanese comics, it works surprisingly well on Western comics as well. We include examples of this in Fig. 10, where we show predictions on Western comics from the Golden Age of Comic Books, which were not part of the training data. Beyond English texts, Magiv3 can still generalize to a degree–for example, it can successfully predict panels, detect and cluster characters for Chinese/Korean comics etc. However, it has poor text detection and no OCR performance on these langauges as one might expect. B. More on character grounding As mentioned in Sec. 4.2 of the main paper, here we explore an alternative, training-free, and zero-shot method for obtaining pre-grounded captions for manga panels. In fact this method is leveraged to acquire training data for Magiv3, as alluded to in Sec. 7 of the main paper. Finally, we discuss the inherent limitations of this technique. The core idea behind this approach is simple yet effective. First, we detect all characters in a manga page using an existing model

FAQs

What is the main advantage of Magiv3 over Florence-2 in caption-to-phrase grounding?

Magiv3 provides context-specific phrase grounding with a unified architecture, enabling it to ground characters in a structured way. Florence-2 instead behaves like caption-derived class-label prediction and struggles to meaningfully ground characters in the given captions.

How does Magiv3 avoid threshold tuning that affects earlier methods?

Magiv3 relies on a next-token prediction paradigm, so it does not require ad-hoc threshold adjustments to filter low-confidence predictions. Earlier approaches can depend on sensitive hyperparameters near decision boundaries.

How are pre-grounded captions for manga panels obtained in the training-free character grounding method?

Characters are detected to produce bounding boxes within panels, each box is assigned a unique identifier via visual prompting, and the prompted panel image is sent to GPT-4o-2024-08-06 with an instruction to generate captions linking IDs to character-referring phrases in a parseable format.

Aug 26, 2026

Uploaded by Ethan Miller