Overview
Our client is an automotive manufacturer in Japan that develops Advanced Driver Assistance Systems (ADAS). It wanted better object detection and quality analysis for the cameras in these systems, including objects the models had rarely seen before. Our task was to build a high-quality annotated dataset of road scenes and train the detection models on it.
Solution
We wrote detailed annotation guidelines and set up the infrastructure for data collection. In CVAT our annotators labeled more than 1.5 million objects in over 20 road-environment categories, including vehicles, pedestrians, roads, road markings and background. Polylines and polygons captured the fine geometry of lanes and surfaces.
The annotated images were used to train deep neural networks for object detection. Our machine learning engineers wrote custom Python scripts and applied MLOps practices across the training pipeline, so data preparation, training and evaluation run as one repeatable process. The project also covered general object detection, labeling that links objects across cameras, and qualitative analysis of model errors, with remote and on-site team members working together.
Technology Stack
Features
- Large-scale annotation: more than 1.5 million objects across 20+ classes, labeled to written guidelines with review steps.
- Detailed labeling: polylines, polygons and segmentation, with lane-specific attributes recorded for each lane line.
- Custom Python scripts: scripts that distribute the output and automate data processing.
- Deep neural network training: the annotated images train the object detection models used in the client's ADAS work.
- MLOps: tools and practices for repeatable training, versioning and deployment of models.
- Continuous data loop: a pipeline from image capture through manual labeling to regular model retraining.
- Room to grow: the infrastructure was extended to further tasks and models during the project.

Outcome
- A dataset of more than 1.5 million annotated road objects, with written rules for every class.
- Object detection models trained on that dataset and ready for the client's own ADAS integration and testing.
- Review steps that catch label errors before they reach training.
- A data and training pipeline the client can reuse for other object detection applications in driver assistance.
road objects annotated
object categories in the labelling scheme
project timeline
How We Build Perception Datasets for Driver Assistance
A driver assistance model is only as reliable as the dataset behind it, and most of the work in an ADAS project sits in the data loop:
- Ontology first. Classes, attributes and occlusion rules are written down with visual examples before labelling starts. Lane lines and road surfaces get their own geometry rules: polylines for lines, polygons for areas.
- Pre-labelling. A detection model trained on earlier batches proposes boxes and masks in CVAT. Annotators correct them instead of drawing from scratch.
- Layered review. A second annotator checks each frame, and a reviewer samples every batch against a gold set. Errors are counted by class and by type: missed object, wrong class, loose boundary.
- Scripted consistency checks. Python scripts flag impossible results, such as boxes outside the image, duplicate instances of one object or a lane line that jumps between frames.
- Training and error mining. Models are trained in PyTorch. Their false positives and misses on new footage go back to annotators, which is how rare objects get into the dataset.
- Versioned releases. Every dataset version is frozen and tied to the models trained on it, so a change in model behaviour can be traced to the data that caused it.
The aim is a loop that runs every week without heroics: new footage in, corrected labels out, a retrained model measured against a fixed test set.
Why Label Quality Is the Hard Part
Public driving datasets show how easily labelling errors survive review. A 2025 re-audit of pedestrian labels in the KITTI benchmark found that at least 18% of pedestrian labels were missing or inaccurately placed in the original ground truth. The same study reports that even the best automatic error-detection methods still miss up to 66% of label errors. KITTI is a long-established, heavily studied benchmark, so private datasets have little reason to be cleaner.
The errors cluster where safety matters most: small and distant objects, pedestrians partly hidden by vehicles, night scenes, glare and rain. An unknown object on the road is rare in the training data by definition, which makes unfamiliar obstacles the hardest category both to label and to test.
This is why some work stays manual. Senior annotators and engineers decide how ambiguous cases are labelled, review hard scenarios by hand and build test sets for rare conditions on purpose instead of sampling them at random. Automated checks catch the obvious mistakes. People catch the ones that change model behaviour. More on this work: data annotation services and computer vision development.