comma.ai Video Compression Challenge
Ongoing video-compression research, with three submissions that have appeared on comma.ai’s leaderboard.
I’m working on comma.ai’s lossy video compression challenge: making a driving video smaller while preserving what the challenge’s segmentation and motion-estimation models see. That means testing both the file size and the effect of compression on those models.
I use comma-lab for experiments, decoders, evaluation tools, and research records. tac contains reusable training, compression, and validation code. I develop across Apple Silicon and Linux GPU environments and check submission archives with the challenge’s evaluator.
Leaderboard submissions
My submissions #107, #110, and #140 have all appeared on the official leaderboard:
- #107, Apogee: an HNeRV-based representation with a deterministic decoder.
- #110: per-frame-pair corrections selected against both segmentation and pose distortion, stored with fixed Huffman coding.
- #140: semantic edits and pose compensation, combined with improvements to lossless coding and the archive format.
These submissions build on credited public work. #110 uses #101’s HNeRV representation; #140 builds on the learned renderer and pose components in #130 and #135, including #133’s contributions. The pull requests describe my optimization, selection, coding, and validation work.
For a more visual introduction to related research, The Witness Machine is an interactive notebook about allocating compression error around what a perception model needs. Its demonstrations are separate from official challenge scores.
The research is ongoing. These leaderboard results apply to the challenge’s fixed video and evaluators.
Public submission reports · 600 samples each
What goes into the score?
The challenge charges for file size and for changes in two perception models’ outputs. Lower is better.
PR #140: approximate score 0.1480.
- Archive size
- 180,002 B
- Original video
- 37,545,489 B
- Score, approximately
- 0.1480
100 × SegNet distortion + √(10 × PoseNet distortion) + 25 × archive/original size. Recomputed from the rounded components in PR #140. Evaluation: CUDA · Tesla T4. Hardware differs between reports, and these values are not a live ranking.
Three submissions along the way
I’ve been working on this lossy video compression challenge, and these three submissions have appeared on the leaderboard at different points. The dates below are when I opened each pull request, not leaderboard entry dates.
May 4, 2026 · CUDA · Tesla T4
PR #107: Apogee
A submission built on the public HNeRV lineage, with a deterministic decoder wrapper and archive checks tied to the evaluation report.
May 20, 2026 · CPU · PyTorch
PR #110: Fixed-Huffman selector
I added a per-frame-pair perturbation selector and a compact Huffman-coded index stream to the frozen PR #101 representation. The underlying HNeRV decoder and weights are credited upstream work.
September 3, 2026 · CUDA · Tesla T4
PR #140: Semantic joint context mixing
I priced segmentation edits against their pose error, re-solved the pose coefficients, and reduced the lossless representation. The learned renderer and pose-carrier design build on PRs #130 and #135, with #133 in that lineage.
The archive sizes and SHA-256 hashes were checked against the downloaded releases on September 28, 2026. The scores are from the public evaluation reports; I have not re-run those evaluations for this exhibit. Archive verification record.
A frame and the model’s segmentation


This reference pair comes from Witness Machine. Its cached segmentation was produced on macOS CPU and is a visual diagnostic, separate from the submission reports above. It does not show a reconstruction from any of these three archives.