DirCast: Live Esports Directing and Commentary
with a Single Streaming Video-Language Model

TL;DR. We propose DirCast, a single streaming video-language model that receives the raw camera of every player in a live Dota 2 match and, every second, both chooses which camera goes on air and commentates over its own choice.

Video

DirCast running on two held-out professional matches. Every cut and every word is the model's own output; nothing is edited. The panel on the right shows the ten hero cameras, and the bars show the hold-or-switch gate and the speak probability against their rate-controlled thresholds.

Control room

Step through a held-out match second by second. On the left is what DirCast puts on air and says; next to it is what the human crew aired and what the casters said. Below them, the ten hero cameras: DirCast's choice is lit in orange and the human director's is outlined in dashed blue. The gauges show the hold-or-switch gate and the speak probability, each against its rate-controlled threshold. Click any second on the timeline or any camera to look at it.

❚❚ MATCH 1MATCH 2 0:00

DIRCAST ·

HUMAN CREW ·

GATE g_t  τ

SPEAK p_t  τ

DIRCAST

HUMAN

KILLS

Abstract

A live esports broadcast needs a director who chooses which camera goes on air and a caster who talks over that choice, every second and without knowing what comes next. Streaming video language models have learned to commentate, but only over a broadcast that a human director has already produced. We study live broadcast production, in which a model receives the raw camera of every player and must both direct and commentate from the past alone. We present DirCast, a single streaming video language model that does both. Every second it scans all cameras at low resolution, decides with a hold-or-switch director whether and where to cut, and reads the chosen camera at high resolution before it speaks. Its context keeps what went on air and what was said and lets the other views expire, which bounds the cost of a stream of any length. We build a dataset of 27 professional Dota 2 matches with a camera for every hero, the human director's choices and the casters' commentary. DirCast directs better than rule-based and zero-shot directors including GPT-5, and its commentary wins 29.2 percent of comparisons against GPT-4o under the Proact-VL protocol, against 19.4 percent for the strongest of eleven streaming commentators that watch the human broadcast.

Method

• One stream. Directing and commentary run in one causal model, in the order a human crew works: look at all cameras, pick one, then talk about the picture that went on air.
• Hold or switch. A gate \(g_t\) decides whether to leave the current shot and a pointer \(\pi_t\) decides where to go, so holding is the default: \(P(c_t{=}k) = (1-g_t)\,\mathbf{1}[k{=}c_{t-1}] + g_t\,\pi_t(k)\).
• The program is the memory. Low-resolution views expire after \(W\) seconds, while the on-air views, the events and the words stay, so the cost per second does not grow with the stream.
• Training. Perturbed program rollouts teach the model to recover from its own wrong shots, and event-anchored targets teach it that kills are worth showing.
• Decoding. The thresholds for cutting and speaking follow the model's running rates toward those of human crews, \(\tau_t = \tau_0 + \gamma\,(\bar r_t - r^{*})\).

Results

All numbers are on the full test split of four unseen matches. Every model runs freely and only sees its own past decisions. Best results are in bold, second best underlined.

Directing

A cut is correct if it falls within two seconds of a human cut and goes to the same feed. The human crew cuts 7.9 times per minute. DirCast is the mean of three seeds.

Commentary

Proact-VL protocol, judged by GPT-5. CC is the win rate against GPT-4o. † Fine-tuned on our training split. The baselines comment on the finished human broadcast, while DirCast comments on the program it directs itself.

Examples

Kills from the test matches that happen away from the shot on air. The human broadcast shows a hero who is not involved, while DirCast cuts to the fight within a second.

Citation

@article{gao2026dircast,
  title   = {DirCast: Live Esports Directing and Commentary with a Single Streaming Video-Language Model},
  author  = {Gao, Zixian and Xiu, Lixin and Luo, Penghao and Cao, Xiangcheng and Cai, Ziqi and Li, Zhen and Wu, Erwin and Zhang, Kaipeng},
  journal = {arXiv preprint arXiv:2610.31734},
  year    = {2026}
}