Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs

Jul 16, 2026

Upload by Rowan

Deploying AI-based visual inspection in manufacturing is difficult because requirements evolve, new defect types emerge, and large labeled datasets are rarely available. The proposed answer-conditioned chain-of-thought (CoT) distillation rapidly adapts small vision-language models (VLMs) to new industrial tasks using minimal labeled data. A frontier VLM generates label-justified visual explanations, and a 3B LoRA fine-tuning step trains on reasoning-augmented examples, improving performance even when teacher accuracy is low.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

Introduction

Related Work

Method

Prompt Design

arXiv :2607 . 10666v1 [ cs .CV] 12 Jul 2026 Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs Shubham Rao Entropy AI Research Labs Private Limited [director@entropyresearch.ai](director@entropyresearch.ai) [https://entropyresearch.ai](https://entropyresearch.ai) Abstract Deploying AI-based visual inspection in manufacturing is hard because requirements change often, new defect types appear, and large labeled datasets are rarely available. We propose answer-conditioned chain-of-thought (CoT) distillation for rapidly adapting small vision-language models (VLMs) to new industrial tasks using minimal labeled data. A frontier VLM receives each training image along with its correct label and generates a justified visual explanation. A 3B-parameter model is then fine-tuned on these reasoning-augmented examples via LoRA. By conditioning on correct answers, we ensure all training reasoning is directed toward the correct conclusion, which is critical because frontier models score as low as 24.1% on our hardest task. We validate on four industrial classification tasks spanning three image modalities using only 18 to 30 labeled images per task. Across 4 seeds per task (32 training runs), our method outperforms direct fine-tuning on all 16 seed-task combinations, with mean improvements of +1.7 to +4.4 percentage points. A controlled equal-budget experiment confirms the improvement comes from reasoning quality, not additional training steps. An unconditioned baseline demonstrates that without answer-conditioning, wrong reasoning degrades performance by 17.8 percentage points. On weld radiograph classification, the fine-tuned 3B model outperforms GPT-4.1 by 10.0pp using just 24 training images. 1 Introduction Visual quality inspection is central to manufacturing. Every production line needs to identify defects, classify materials, or verify compliance with standards. These requirements change often. Each change demands anew or retrained vision model. Traditional approaches rely on CNNs trained on thousands of labeled images [2] . Collecting this data is expensive. In many settings, a factory has fewer than 30 labeled examples and needs a working classifier within days, not months. Large VLMs like GPT-4.1 can interpret images without task-specific training. But their accuracy on specialized industrial tasks is poor. On concrete aggregate grading (9 classes, DIN 1045 [4]), GPT-4.1 achieves 29.6% few-shot. Gemini 2.5 Pro achieves 24.1% . These models cannot be deployed on-premises due to cost and data privacy constraints. Small VLMs (1–7B parameters) [14] fit on edge hardware and can be fine-tuned with LoRA [6] . But standard fine-tuning on 18–30 images maps images to labels without transferring domain knowledge. The model memorizes rather than learns. CoT distillation [5] trains smaller models on reasoning-augmented data from larger models, but existing work targets text-only settings. STaR [17] and Video-STaR [19] use answer-conditioned reasoning in text and video settings, but not for VLMs where the teacher itself performs poorly. We propose answer-conditioned CoT distillation for VLMs. A frontier model receives each training image with its correct label and explains why that label is correct. A small VLM is fine-tuned on these reasoning-augmented pairs via LoRA. The frontier model is not classifying. It already knows the answer. It is explaining what visual features justify that answer. This is critical because frontier model accuracy ranges from 24.1% to 91.1% across our tasks. If we asked it to classify and explain, up to 76% of training data would contain incorrect reasoning. We show experimentally that unconditioned CoT destroys performance (−17.8pp) when the teacher is mostly wrong. Our contributions: 1. We apply answer-conditioned CoT distillation to VLMs for industrial few-shot classification, showing consistent improvement across 4 tasks, 3 modalities, a

FAQs

为什么制造业视觉检测在实际落地中难以完成？

需求经常变化、新缺陷类型会出现，而大规模标注数据通常难以获得。

答复条件（answer-conditioned）的 CoT 蒸馏是如何工作的？

前沿 VLM 在看到训练图像时输出正确标签及其理由解释；随后用带有推理增强样本的数据对小型 VLM（例如 3B）进行 LoRA 微调，并通过“正确答案条件”把推理对齐到正确结论。

实验结果表明该方法带来的收益来自哪里？

受控的等预算实验表明提升来自推理质量，而不是额外训练步骤；未进行 answer-conditioning 的基线在教师推理错误时会显著降性能。

Jul 16, 2026

Upload by Rowan