ML/Computer Vision-Powered Data Annotation for Sports
AI Development •Data Annotation •Data Science •Machine Learning •Sports

ML/Computer Vision-Powered Data Annotation for Sports

We worked with a sports analytics firm on player and event tracking from broadcast camera feeds. Computer vision places players and actions on field coordinates, annotators correct the tracks, and coaches and scouts get data for tactics and recruitment.

Facing a similar challenge? Tell us about the task, and we will show how we would build it for your business.

Build Something Like This for Me

Free 30-min call · One-page solution outline within three business days · No commitment

Overview

Our client is a European sports analytics firm that uses AI in performance analysis and player recruitment. It wanted to analyze player movement and key game events straight from broadcast camera feeds, without installing extra hardware. The goal was to track players, record the field coordinates of important actions and give coaches and scouts data they can use alongside their own judgement.

Solution

We put together a team covering machine learning, computer vision, data annotation and MLOps. The platform tracks the players visible in the main broadcast camera and maps their positions to field coordinates, so analysts can study positioning, movement and formations. Annotators correct broken tracks and identity switches.

Event tracking marks the field coordinates where specific actions happen, which supports detailed performance analysis. We built custom annotation tools to label game footage and produce training data for the models. Processing runs on AWS.

Features

Player tracking:

  • Tracks the players visible in the main broadcast camera and maps them to field coordinates
  • Analyzes formation, positioning and movement patterns
  • Human correction of broken tracks and wrong identities

Event analysis:

  • Marks the field coordinates where key actions and events occur
  • Calculates success rates for particular player events
  • Builds datasets of action coordinates

Analytics for coaches and scouts:

  • Data on team performance and game strategy
  • Player metrics that support recruitment work
  • Match data available for analysis soon after the game

Machine learning:

  • Activity recognition models
  • Training on large labeled datasets
  • Regular retraining as corrected data comes in

Custom annotation tools:

  • Efficient annotation of game footage
  • Training datasets checked by reviewers
  • Built to handle large volumes of annotation work

Cloud deployment:

  • Hosted on AWS
  • Connected to the client's systems through APIs
  • Central storage of tracking and event data

MLOps practices:

  • Automated CI/CD pipelines for deployments
  • Docker for containerization
  • Kubernetes for orchestration
fw-image

Outcome

  • Coaches can base tactical discussions on positional and event data, not only on video.
  • Scouts get player metrics to add to their own evaluation of a player.
  • The platform works from existing broadcast feeds, so no extra hardware is needed at the venue.
  • Corrected tracks flow back into training, so each model version starts from better data.
22

players per match the tracking is built to follow, with human correction

0

extra hardware in the stadium: tracking runs on the broadcast feed

8 months

project timeline

How We Build Tracking From Broadcast Video

Broadcast footage was not filmed for analytics. The camera pans, zooms and cuts to replays, so a tracking system has to recover the pitch before it can place a player on it. Our pipelines for this class of problem share the same stages:

  • Shot filtering. Replays, close-ups and graphics are detected and excluded, so only the main tactical camera feeds the tracker.
  • Detection. A detector trained in PyTorch finds players, referees and the ball in each frame. The ball gets separate handling because it is small, fast and often blurred.
  • Camera calibration. Pitch lines and markings are detected in every frame to estimate the camera homography, which maps image pixels to field coordinates.
  • Tracking and re-identification. Detections are linked into tracks across frames. Appearance features, team colours and jersey numbers reconnect a player after an occlusion or a camera cut.
  • Event spotting. A separate model marks when actions such as passes and shots happen, and each event is placed at the field coordinates of the player involved.
  • Human correction. Annotators fix broken tracks and wrong identities in a review tool, and their corrections become training data for the next model version.

Each stage is evaluated on its own. When final accuracy drops, the metrics show whether calibration, detection or identity caused it, instead of leaving the team to guess.

What Makes Broadcast Tracking Hard

Placing every player on a minimap with the right team and identity from a single moving camera is still an open research problem. The SoccerNet benchmark introduced it as the Game State Reconstruction challenge in 2024. According to the SoccerNet 2024 challenge results, 16 teams took part and the best method raised the GS-HOTA score on the 2024 challenge set from 23.36 for the baseline to 63.81, on a scale where 100 would mean every player located and identified correctly. The organizers write that this task may still require further investigation, while more mature tasks such as ball action spotting are approaching industrial use.

The causes are familiar to anyone who has worked with broadcast footage. Players leave the frame and come back. Jersey numbers are readable only in some frames. Teammates look almost identical from the main camera. Calibration fails when few pitch lines are visible, for example in a crowded penalty area.

For this reason, the outputs that feed recruitment and tactical decisions still get human review: identity switches, events in crowded areas and anything that happens outside the camera view. We measure where the model is reliable and route the rest to people. See computer vision development and our sport and wellness work.

Other cases

AI Agent Platform With Governed Model Access: Many Agents, One Runtime

AI Agent Platform With Governed Model Access: Many Agents, One Runtime

Read more ›
AI Candidate Screening That Checks CV Claims Against Public Evidence

AI Candidate Screening That Checks CV Claims Against Public Evidence

Read more ›
ChatGPT-Powered Content Generation Platform

ChatGPT-Powered Content Generation Platform

Read more ›
AI Data Analyst in Microsoft Teams: Answers From Live Operational Data

AI Data Analyst in Microsoft Teams: Answers From Live Operational Data

Read more ›
AI Data Verification Layer for Property Data: Rules First, a Person on Borderline Cases

AI Data Verification Layer for Property Data: Rules First, a Person on Borderline Cases

Read more ›
AI Delegation Agent for Executive Teams: Assignments, Statuses and Scores in One Loop

AI Delegation Agent for Executive Teams: Assignments, Statuses and Scores in One Loop

Read more ›
AI Image Annotation Solution for Medical Assessment

AI Image Annotation Solution for Medical Assessment

Read more ›
ML/Computer Vision + Data Annotation for ADAS

ML/Computer Vision + Data Annotation for ADAS

Read more ›
AI/ML-Driven Telecom Network Optimization

AI/ML-Driven Telecom Network Optimization

Read more ›
Custom GenAI Medical Drug Search Accelerator

Custom GenAI Medical Drug Search Accelerator

Read more ›
OCR Retail Receipt Data Extraction Tool

OCR Retail Receipt Data Extraction Tool

Read more ›
Computer Vision + Image Annotation for Insurance

Computer Vision + Image Annotation for Insurance

Read more ›
AI Presale Assistant for a Marketing Agency: From Meeting Transcript to Sourced Draft

AI Presale Assistant for a Marketing Agency: From Meeting Transcript to Sourced Draft

Read more ›
AI Pricing Engine for an Equipment Distributor: The Model Proposes, a Person Approves

AI Pricing Engine for an Equipment Distributor: The Model Proposes, a Person Approves

Read more ›
AI Trade Contractor Sourcing: Verified Profiles and Pre-Screening for a Home Builder

AI Trade Contractor Sourcing: Verified Profiles and Pre-Screening for a Home Builder

Read more ›
Cinema Scheduling and Pricing Optimization: A Week of Showtimes, Computed

Cinema Scheduling and Pricing Optimization: A Week of Showtimes, Computed

Read more ›
Computer Vision + ChatGPT-Powered Recommendations

Computer Vision + ChatGPT-Powered Recommendations

Read more ›
Data Annotation for AI-Driven Sports Analytics

Data Annotation for AI-Driven Sports Analytics

Read more ›
Generative AI Bot for Customer Service

Generative AI Bot for Customer Service

Read more ›
Generative AI Solution for Code Compliance

Generative AI Solution for Code Compliance

Read more ›
SaaS Marketing Content Generation Platform for Healthcare

SaaS Marketing Content Generation Platform for Healthcare

Read more ›
Interactive Chatbot with an Animated Avatar

Interactive Chatbot with an Animated Avatar

Read more ›
Managed Video Annotation Pipeline for a Client's Own Human-Motion Footage

Managed Video Annotation Pipeline for a Client's Own Human-Motion Footage

Read more ›
ML/AI Technical Due Diligence for Startup Acquisition

ML/AI Technical Due Diligence for Startup Acquisition

Read more ›
OCR Solution for Invoice Processing

OCR Solution for Invoice Processing

Read more ›
Property Management Platform With an AI Voice Assistant for Multiple HOAs

Property Management Platform With an AI Voice Assistant for Multiple HOAs

Read more ›
AI-Powered Platform for Real Estate

AI-Powered Platform for Real Estate

Read more ›
AI Assistant for Report Generation & Visualization

AI Assistant for Report Generation & Visualization

Read more ›
Sitecore Migration

Sitecore Migration

Read more ›
ML/AI Technical Due Diligence for SporTech

ML/AI Technical Due Diligence for SporTech

Read more ›
Voice Interview to Social Posts: AI Content in the Professional's Own Voice

Voice Interview to Social Posts: AI Content in the Professional's Own Voice

Read more ›

Let's talk about your project

We'll help you harness AI and ML technologies to create innovative software solutions

File requirements: pdf, doc, docx, rtf, ppt, pptx — max 15 MB

No file selected