Spatial Reasoning from Synthetic Scenes

Abstract

Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities across a wide range of visual tasks, yet their performance often degrades in specialised domains such as Remote Sensing, where spatial reasoning in Visual Question Answering (VQA) remains particularly difficult. This limitation is primarily driven by a scarcity of high-quality training data, which comes from the rarity of complex real-world events and the labour-intensive process of annotating spatial relationships. To address this, this thesis proposes a synthetic data generation pipeline based on generative inpainting, using parked aircraft as a case study. By inserting synthetic objects into real remote sensing backgrounds, the pipeline creates complex spatial scenarios while preserving realistic context. Because the scene layout is fully controlled, it can also automatically generate grounded image-question-answer triplets together with Chain-of-Thought (CoT) reasoning traces for each answer. To measure this spatial reasoning, we construct an extensive evaluation benchmark on real overhead imagery whose questions span five question domains, from single-object properties to multi-object relational reasoning, which is necessary to properly test complex spatial reasoning in VQA. We fine-tune three open-source LVLMs from different model families on the synthetic data and evaluate them against generalist and domain-specific baselines on this benchmark. Fine-tuning on the synthetic data improves spatial reasoning across all tested domains and across all three models, raising the strongest model, Qwen3.5 9B, from 43.4\% to 71.9\% accuracy, a gain of 28.5 percentage points (pp) over its zero-shot baseline. The consistency of this improvement across architectures indicates that it comes from the data itself rather than from any single model. Adding CoT supervision raises performance further, to a best of 75.5\% (a 3.6 pp increase over standard fine-tuning, and 32.1 pp over the zero-shot baseline). The gains are not uniform: intrinsic orientation (distinguishing the front of an aircraft from its back) remains the lowest-scoring domain for every fine-tuned model, and performance declines as the number of aircraft in a scene grows. Overall, these findings show that synthetic data can serve as an effective alternative or extension to real remote sensing VQA data, helping to address the data scarcity present in this domain, and because the pipeline was built to be modular, its components hold potential beyond this case study, possibly extending to other objects and tasks.