Reasoning-Enhanced Vision-Language Models for Explainable Industrial Anomaly Detection

Abstract

Industrial quality control needs inspection systems that are accurate, explainable, and adaptable to new product categories with few labelled data. Traditional deep-learning detectors, such as memory-bank, normalising-flow, and reconstruction-based methods, reach near-perfect AUROC on the established MVTec AD benchmark, yet they emit only a binary defect verdict and a heatmap. They give no human-readable account of why a sample was flagged or what type of defect it carries, and that lack of interpretability is a primary obstacle to deploying automated inspection in safety- and audit-critical manufacturing.

This thesis asks whether an open vision-language model (VLM), fine-tuned on structured reasoning traces and refined with reinforcement learning, can deliver explainable anomaly detection that is competitive with the state of the art, on products it was not trained on and without per-product retraining or reference images. We build an end-to-end framework that fine-tunes the open-weight Qwen2.5-VL family on structured chain-of-thought reasoning traces. Each trace follows a six-phase template (Framing, Scan, Focus, Evaluate, Alternatives, Decide) and emits an XML output with a free-text rationale, a defect-type tag, a spatial location, and a binary verdict. Using Gemini 2.5-Flash we generate 14,472 such traces from the Real-IAD multi-view dataset, released here as the AnomalyThink dataset, and partition them into product- and class-stratified splits of 6,000 for supervised fine-tuning, 4,236 for GRPO, and 4,236 held out.

We run a controlled four-factor comparison of fine-tuning configurations across model size (3B versus 7B), vision-encoder freezing, supervised-data scale (6K versus the full 14.5K union), and training epoch (up to four), with the unfrozen encoder evaluated only at the larger scale. This is followed by Group Relative Policy Optimisation (GRPO) with an unweighted two-function reward scoring four bounded signals: format, verdict, semantic type, and location. Evaluation uses the DS-MVTec (1,670 samples) and VisA (2,141 samples) subsets of MMAD, together with a held-out Real-IAD split.

Our main result comes from a three-stage pipeline in which each stage feeds the next: supervised fine-tuning, GRPO, and a second supervised stage trained on data the GRPO model produced. Supervised fine-tuning on the 6,000-trace dataset reaches 80.16 % balanced accuracy on DS-MVTec and 64.78 % on VisA. GRPO with our reward functions lifts this to 82.73 % and 70.39 %. We then roll that GRPO model out at k = 8 over all 10,236 training images, which is the 6,000-image SFT split plus the 4,236-image GRPO split. We keep the traces it gets right and have a teacher correct or rewrite the rest. The new corpus is rebuilt on the 6,000 SFT images. Training the base Qwen2.5-VL-7B on it reaches 82.80 % and 72.07 %, the best result on that backbone, and beats the released IAD-R1 model on both subsets (81.92 % and 71.34 %) under the same evaluation harness. We repeat the same recipe end to end on a second backbone, LLaVA-OneVision-7B-SI. There the curated corpus reaches 87.32 % and 72.65 % at the selected epoch against 87.66 % and 72.38 % for SFT plus GRPO, and 74.29 % on VisA by epoch 4. On both backbones the supervised stage matches the reinforcement-learned one and exceeds it on VisA. The finding is therefore not specific to one model family. The last stage is a supervised one. What GRPO gave us was not the final model but the data we used to train it.

GRPO on its own helps when the supervised baseline is weak. Applied to the earlier supervised baseline it raises balanced accuracy to 82.73 % on DS-MVTec, 70.39 % on VisA, and 80.87 % on the held-out Real-IAD split, exceeding IAD-R1 on DS-MVTec. Applied on top of the stronger curated dataset, GRPO did not improve binary accuracy in our experiments. Reward kept rising while balanced accuracy stayed 1.6 to 4.0 points below the supervised initialisation. This held for all three advantage estimators we tried, within a single 120-step probe on one initialisation. Two further results follow. Within our data, the 6,000-trace split beats the full 14.5K union by 8.5 balanced-accuracy points on DS-MVTec at matched epochs, a data-count effect rather than a trace-length one. And freezing the vision encoder consistently beats unfreezing it, most clearly on the harder VisA benchmark.

The framework is reproducible on commodity hardware, two RTX A6000 GPUs with ZeRO-3 CPU offloading, and yields a single 7B model that detects, localises, and explains industrial defects in natural language, with no per-product retraining and no pixel-level masks at inference.