Dynamic Bidirectional Pattern Memory Inference-Time Gating for Clinical NLP Production-Scale Empirical Characterisation

Jul 16, 2026

Upload by Anda

Dynamic Bidirectional Pattern Memory (DBPM) studies inference-time gating for a production-scale clinical NLP pipeline that couples a generator with a verifier. Over 167,034 PMC-Patients narratives, DBPM adds a lightweight memory learned at deployment to filter extractions using evidence from the verifier’s verdict stream. Four key results show that verifier-rejection-derived learning fails at full scale, while ontology-based filtering works, ontology- and evidence-aligned QA gating succeeds, and selectivity depends on testing the same evidence the verifier weighs.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages39

Reading Time1 h 38 min

Introduction

Dynamic Bidirectional Pattern Memory and pipeline design

Central research question and gating selectivity

Experimental characterisation and findings

Learning from verifier rejections at full scale

Ontology-based filtering

Question-answering filter variants

Evidence alignment regularity

Dynamic Bidirectional Pattern Memory: A Production-Scale Empirical Characterisation of Inference-Time Gating in Clinical NLP⋆ Ali H. Lazema,b,∗ (Corresponding Author), William Teahana (Co-Author) a School of Computer Science and Engineering, Bangor University, Bangor, Gwynedd, LL57 2DG, United Kingdom b University of Thi-Qar, Nasiriyah, 64001, Iraq arXiv :2607 .00870v 1 [ cs .CL] 1 Jul 2026 ARTICLE INFO Keywords: multi-agent pattern memory inference-time gating clinical natural language processing large language model verification generator-verifier architecture AB STRACT We study inference-time pattern-memory gating in a production-scale clinical natural language processing (NLP) pipeline. The pipeline pairs a generator (Llama-3.3 70B) that proposes extractions with a verifier (MMed-Llama-3.1 70B) that accepts or rejects them, over 167,034 PMC-Patients narratives, and we add a lightweight memory that learns at deployment time which extractions to filter, so the verifier need not re-examine candidates already seen to fail. We report four findings. First, learning these filtering rules directly from the verifier’s rejections did not work at full scale: the relation-extraction filter ended up empty even though the pipeline logged 785,797 rejected candidates, because the rejections were spread too thinly across too many distinct forms to accumulate. Second, a simpler rule based on a fixed clinical ontology produced the same filtering without relying on the verifier at all, capturing 49,734 ontologyviolating relations on a held-out 5,000-patient set. Third, we tested five versions of the questionanswering filter; four failed for distinct and instructive reasons, and the fifth succeeded by checking whether a patient’s extracted clinical entities actually support the question being asked; on the categories where it applies, it was 1.84 times more likely to flag an answer the verifier would reject than one it would accept. Fourth, across all five versions one pattern held: a filter is selective only when it tests the same evidence the verifier itself weighs, not when it tries to imitate the verifier’s output. Taken together, these findings give a practical, transferable result for any generator-verifier pipeline: the most natural memory design can fail silently at deployment scale, and whether a pre-generation gate is selective is decided before any engineering effort, by whether its signal probes the question the verifier itself answers. Throughout, the system flags suspect extractions rather than deleting them, so every decision stays visible for clinical review. All code and test artefacts are released openly. 1. Introduction Large language models have transformed clinical natural language processing, enabling extraction of named entities, relations, and question–answer pairs from clinical narratives at scales unattainable with prior rule-based or fine-tuned-classifier pipelines. A growing body of work pairs a generator model with a verifier model that scores or accepts the generator’s outputs (Madaan et al., 2023; Shinn et al., 2023; Yao et al., 2023), adding an inference-time correctness check on top of instruction tuning and retrieval. This pattern is attractive in clinical settings, where an unverified extraction can carry a factual error into a patient record. At the scale of clinical informatics corpora, however, where hundreds of thousands of narratives are processed without model retraining, inference-time verification becomes the dominant computational expense. This motivates a mechanism that avoids invoking the verifier on candidates the pipeline has, in effect, already seen fail: an inference-time memory that accumulates verdict evidence as the pipeline runs and gates future candidates accordingly, without retraining either model. The design space for such a memory is large, and which choices actually yield selective gating, gating that flags the candidates a verifier would reject while leaving t

FAQs

What problem does DBPM address in a generator-verifier clinical NLP pipeline?

DBPM reduces the need to call the verifier on candidates that have already been judged as failures, by using an inference-time memory that accumulates verdict evidence and gates future extractions without retraining either model.

Why did learning filtering rules directly from the verifier’s rejections fail at full scale?

At full scale, the rejections were spread across too many distinct forms, so the relation-extraction filter ended up empty despite the pipeline logging many rejected candidates.

What determines whether a pre-generation gating signal is selective?

A filter is selective only when it directly tests the same evidence that the verifier itself uses; signals that probe different evidence do not separate verifier-rejected from verifier-accepted candidates.

Jul 16, 2026

Upload by Anda