Teaching AI to Spot Vehicles When Real Data Is Scarce

Military vehicle detection has a data problem: real images are expensive, sensitive, and rare. So we asked a strange question: What if we just rendered our own vehicles in Blender? We built a synthetic image pipeline, trained object detectors on the results, and learned exactly when fake data pays off, and when it doesn't.

By Julian Cornel Berndt, Data Scientist, Defence, SitaWare Insight

The problem: not enough tanks

Object detection models are hungry. They want heaps of labelled images to learn from. In defence, you rarely get that. Military vehicles are operationally sensitive, hard to photograph from every angle, in every weather, and distance, and expensive to collect data on at scale. We had 420 real, hand-annotated images to work with. That's not a lot for a transformer-based object detector.

So we asked: can synthetic images fill the gap?

The fix: render your own dataset

We built a fully automated pipeline in Blender and Python that randomly places 3D military vehicle models (trucks, main battle tanks, surface-to-air missile systems) into 3D environments, calculates the correct vehicle tilt via raycasting to adhere to the terrain and then photographs them from randomised angles, distances, and in various lighting conditions. Turrets can be rotated. Weather changes as well. Every image comes with a COCO-format bounding box annotation generated automatically from the scene geometry, no manual labelling needed.

We used Blender's Cycles path-tracer for physically realistic light bounces and shadows, rendered at 1920×1080, and generated 420 synthetic images to match our 420 real ones. Then we trained RF-DETR object detectors (with a DINOv2 backbone) on real-only, synthetic-only, and mixed datasets, and tested everything on 320 real-world images the models had never seen.

The result: it depends how desperate you are

Synthetic data isn't a universal upgrade, its value depends entirely on how much real data you already have.

• At 5% of real data (21 images): adding all the synthetic images more than doubled recall (20% → 41%) and pushed the F1-score from 32% to 57%, while precision actually improved slightly. The model went from missing most objects to finding nearly half of them, with more confidence.

• At 10% of real data (42 images): the same pattern holds. Precision rose from 81% to 86%, recall climbed from 23% to 38%, and F1 improved from 35% to 53%. True positives increased markedly, with only a modest rise in false positives, a strong return for very little real data.

• At 25–50% real data: synthetic data still boosted F1, but the trade-off changed. Precision dropped a bit while recall jumped, meaning the detector got less cautious and started finding more objects, at the cost of a few more false alarms.

• At 100% real data (420 images): the pattern flipped entirely. Adding synthetic data increased precision to 97% but recall collapsed to 48%, and F1 dropped from 76% to 65%. With plenty of real data already, synthetic images made the model more conservative, pickier, not better.