Concept artwork of football footage with player detection boxes on a desktop monitor
Concept artwork · not an app screenshot

Case overview

An in-progress computer vision experiment that samples real video, detects players, reads jersey-number candidates, and links sightings across frames.

StatusBuilding
FocusC# · ASP.NET Core · Apple Vision

01The problem

A jersey number is a small, moving signal. Distance, blur, occlusion, and changing camera angles make it difficult to read consistently. A useful analysis tool needs to preserve the evidence behind a reading so a person can review it.

02From video to evidence

The browser decodes sampled frames and sends JPEGs to a local C# endpoint. Apple Vision detects people using overlapping tiles, then reads enlarged upper-body crops. Each candidate retains its timestamp, crop, raw text, and OCR score. The review workspace lets a user verify readings and export JSON evidence.

03Tracking decisions

A motion-predicted overlap and center-distance baseline links sightings with one-to-one assignment. It rejects large size changes and close competing matches; gaps longer than 2.5 seconds start a new track. Repeated number support requires at least two readings and a margin over conflicting votes.

04Selective AI review

An optional review path prioritizes conflicting, unreadable, and single-reading tracks. It selects one crop per track and caps requests at 1, 3, or 5 per run. AI suggestions remain separate from baseline OCR evidence. Provider inference and its effectiveness are not yet verified in the project notes.

05Evidence so far

A documented 18.8-second clip run with 24 evenly spaced frames produced 276 person sightings, 60 crops with numeric readings, and 62 provisional track fragments. Seven tracks had repeated number support. These observations demonstrate an operating pipeline; they do not establish player identity or tracking accuracy. Synthetic association checks cover movement, missed samples, ambiguity, long gaps, one-to-one assignment, and conflicting OCR.

06Tradeoffs and next steps

Local detection and OCR make the baseline usable without AI calls, but small digits and crowded scenes remain difficult. The tracker has no appearance re-identification or camera compensation, so crossings and camera cuts can fragment tracks or switch identities. Next, evaluate against independently labeled footage and compare a jersey-specific reader with selective AI review.