HAT Super Resolution and PARSeq+CLIP4STR Voting Ensemble for Extreme In the Wild License Plate Recognition

Jul 16, 2026

Upload by Riley

Entry for the ICIP 2026 Grand Challenge on Extreme In-the-Wild License Plate Super-Resolution (XLPSR) achieving 9.73 wECR on the public validation leaderboard. The approach combines a Hybrid Attention Transformer (HAT) 4× super-resolution front-end with an ensemble of two scene-text recognizers, PARSeq-S and CLIP4STR-B. A confidence-weighted per-character soft-voting scheme performs abstention on uncertain positions using τ=0.33, aligning decisions with the +2/−1/0 scoring rule. Runtime targets 1.7 s per sequence on RTX 3090 within Docker limits.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages2

Reading Time5 min

Introduction

Method

Multi-frame fusion

Super-resolution

Two-model OCR ensemble

Character voting with abstention

Training Data

HAT Super-Resolution and a PARSeq+CLIP4STR Voting Ensemble for Extreme In-the-Wild License Plate Recognition Karthik Sivarama Krishnan∗† ∗ Independent Researcher †Ancestry [ks7585@g.rit.edu](ks7585@g.rit.edu) Koushik Sivarama Krishnan∗ ∗ Independent Researcher [koushik.nov01@gmail.com](koushik.nov01@gmail.com) arXiv :2607 .08896v 1 [ cs .CV] 9 Jul 2026 | 10 input frames per sequence | | | --- | --- | | | | | Laplacian variance → sharpest reference | | | | | | ECC affine alignment + pixel-average fusion | | | HAT ×4 super-resolution (6 views) | | | --- | --- | | | | PARSeq-S v2 CLIP4STR-B (32 × 128) (224 ×224) | Per-position [soft-conf. vote](soft-conf. vote) (2:1) + abstention τ=0.33 | | --- | | | | 7-character output (e.g. AB123CD) | Fig. 1. Pipeline. Ten sequence frames are ranked by Laplacian variance; five aligned crops plus a temporally-fused crop are 4× super-resolved by HAT and passed through two scene-text recognisers whose per-position soft-max outputs are combined by a confidence-weighted vote with τ=0 .33 abstention. Abstract—We describe our entry to the ICIP 2026 Grand Challenge on Extreme In-the-Wild License Plate Super-Resolution (XLPSR), which scored 9.73 wECR on the public validation leaderboard. The system pairs a Hybrid Attention Transformer super-resolution (HAT) front-end with an ensemble of two scenetext recognisers (PARSeq-S and CLIP4STR-B) and a confidenceweighted character-voting scheme that abstains on uncertain positions. We treat XLPSR as a recognition task gated by image legibility: the SR step exists to lift characters out of sub-pixel territory, and the asymmetric scoring rule (+2/−1/0) is exploited explicitly through abstention. Our pipeline runs in 1.7 s per sequence on RTX 3090 (max 2.7 s, p99 2.4 s), well under the 60 s/sequence Docker budget. Index Terms—License plate recognition, Super-resolution, Scene text recognition, Ensemble, Abstention. I. INTRODUCTION The XLPSR challenge provides 10 video frames per sequence of moving vehicles with French license plates under extreme in-the-wild degradation: plates as narrow as 12 px, motion blur, JPEG compression, adverse lighting, and occlusion. The task is to predict the plate text. The +2/−1/0 percharacter scoring makes guessing in low-confidence regions negative-EV below P (correct) = 1/3, which we exploit explicitly. Figure 1 summarises the pipeline; the following sections describe each stage. II. METHOD A. Multi-frame fusion Each sample in the blind-evaluation set is supplied as a temporal sequence of already-cropped license-plate images; the public-validation set provides full frames with per-frame bounding boxes which our pipeline crops first. After that branching step the rest of the pipeline is identical: we score every crop by Laplacian variance, pick the sharpest as a reference, align the remaining crops to it with the Enhanced Correlation Coefficient (ECC) algorithm under an affine motion model, and average them pixel-wise into a fused crop. ECC + affine was chosen over optical-flow alternatives because it is robust to the low-contrast conditions that dominate XLPSR crops, requires no ground-truth flow supervision, and is fast enough for CPU-side pre-processing. We carry forward six views per sequence into the recogniser stack: the sharpest reference, up to four additional aligned crops (in input frame order), and the fused crop. B. Super-resolution Each view is upscaled 4 × by Real-HAT-GAN-SRx4 [1], a Hybrid Attention Transformer with six RHAG blocks at embed dim 180, window 16, overlap ratio 0.5, and a PixelShuffle upsampler. We use the public weights without further fine-tuning. Lighter real-time variants such as our earlier SwiftSRGAN [6] would suit tighter latency budgets, but XLPSR’s 60 s/sequence budget lets us prioritise legibility over throughput. SR is the single largest contributor to wECR in our ablation (Table I,+2 .00 over the PARSeq+CLIP4STR no-SR baseline) . C

FAQs

What is the core pipeline used for XLPSR license plate recognition?

The system ranks video frames by Laplacian variance, aligns multiple crops to a sharp reference using ECC affine alignment, fuses them, then performs 4× super-resolution with HAT. Both the super-resolved views and the original-resolution reference are decoded by PARSeq-S and CLIP4STR-B, and results are merged via confidence-weighted character voting with abstention.

Why use multi-frame fusion with ECC affine alignment instead of optical flow?

ECC affine alignment is selected for robustness under low-contrast conditions typical of XLPSR crops, avoids requiring ground-truth flow supervision, and remains fast enough for CPU-side pre-processing.

How does character voting work, and how is abstention applied?

For each character position, soft-max confidence is summed per class across retained recognition tuples, weighting PARSeq twice and CLIP4STR once. The system abstains (emits an underscore) when the winning class soft-max mass falls below τ=0.33, which corresponds to the break-even P(correct)=1/3 implied by the +2/−1/0 scoring rule.

Jul 16, 2026

Upload by Riley