Overview
Our client is a European sports analytics firm that uses AI in performance analysis and player recruitment. It wanted to analyze player movement and key game events straight from broadcast camera feeds, without installing extra hardware. The goal was to track players, record the field coordinates of important actions and give coaches and scouts data they can use alongside their own judgement.
Solution
We put together a team covering machine learning, computer vision, data annotation and MLOps. The platform tracks the players visible in the main broadcast camera and maps their positions to field coordinates, so analysts can study positioning, movement and formations. Annotators correct broken tracks and identity switches.
Event tracking marks the field coordinates where specific actions happen, which supports detailed performance analysis. We built custom annotation tools to label game footage and produce training data for the models. Processing runs on AWS.
Technology Stack
Features
Player tracking:
- Tracks the players visible in the main broadcast camera and maps them to field coordinates
- Analyzes formation, positioning and movement patterns
- Human correction of broken tracks and wrong identities
Event analysis:
- Marks the field coordinates where key actions and events occur
- Calculates success rates for particular player events
- Builds datasets of action coordinates
Analytics for coaches and scouts:
- Data on team performance and game strategy
- Player metrics that support recruitment work
- Match data available for analysis soon after the game
Machine learning:
- Activity recognition models
- Training on large labeled datasets
- Regular retraining as corrected data comes in
Custom annotation tools:
- Efficient annotation of game footage
- Training datasets checked by reviewers
- Built to handle large volumes of annotation work
Cloud deployment:
- Hosted on AWS
- Connected to the client's systems through APIs
- Central storage of tracking and event data
MLOps practices:
- Automated CI/CD pipelines for deployments
- Docker for containerization
- Kubernetes for orchestration
Outcome
- Coaches can base tactical discussions on positional and event data, not only on video.
- Scouts get player metrics to add to their own evaluation of a player.
- The platform works from existing broadcast feeds, so no extra hardware is needed at the venue.
- Corrected tracks flow back into training, so each model version starts from better data.
players per match the tracking is built to follow, with human correction
extra hardware in the stadium: tracking runs on the broadcast feed
project timeline
How We Build Tracking From Broadcast Video
Broadcast footage was not filmed for analytics. The camera pans, zooms and cuts to replays, so a tracking system has to recover the pitch before it can place a player on it. Our pipelines for this class of problem share the same stages:
- Shot filtering. Replays, close-ups and graphics are detected and excluded, so only the main tactical camera feeds the tracker.
- Detection. A detector trained in PyTorch finds players, referees and the ball in each frame. The ball gets separate handling because it is small, fast and often blurred.
- Camera calibration. Pitch lines and markings are detected in every frame to estimate the camera homography, which maps image pixels to field coordinates.
- Tracking and re-identification. Detections are linked into tracks across frames. Appearance features, team colours and jersey numbers reconnect a player after an occlusion or a camera cut.
- Event spotting. A separate model marks when actions such as passes and shots happen, and each event is placed at the field coordinates of the player involved.
- Human correction. Annotators fix broken tracks and wrong identities in a review tool, and their corrections become training data for the next model version.
Each stage is evaluated on its own. When final accuracy drops, the metrics show whether calibration, detection or identity caused it, instead of leaving the team to guess.
What Makes Broadcast Tracking Hard
Placing every player on a minimap with the right team and identity from a single moving camera is still an open research problem. The SoccerNet benchmark introduced it as the Game State Reconstruction challenge in 2024. According to the SoccerNet 2024 challenge results, 16 teams took part and the best method raised the GS-HOTA score on the 2024 challenge set from 23.36 for the baseline to 63.81, on a scale where 100 would mean every player located and identified correctly. The organizers write that this task may still require further investigation, while more mature tasks such as ball action spotting are approaching industrial use.
The causes are familiar to anyone who has worked with broadcast footage. Players leave the frame and come back. Jersey numbers are readable only in some frames. Teammates look almost identical from the main camera. Calibration fails when few pitch lines are visible, for example in a crowded penalty area.
For this reason, the outputs that feed recruitment and tactical decisions still get human review: identity switches, events in crowded areas and anything that happens outside the camera view. We measure where the model is reliable and route the rest to people. See computer vision development and our sport and wellness work.