Self-Play Reinforcement Learning with Search
Agents that reach superhuman play by playing against themselves at scale, often with tree search.
Tier Grades
Tier A: Frontier AI labs
feasible
OpenAI trained Five for 10 months on about 2 million frames every 2 seconds.
Tier B: Indonesian AI labs
not feasible
Ten months of distributed training is out of reach on the evidence.
Tier C: Institution units such as ITB
stretch
Small-scale reproductions on board games are possible, but frontier-scale runs are not.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
not feasible
No delivery use is shown.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeOpenAI Five: batches of about 2 million frames every 2 seconds, trained for 10 months.
- DataOpenAI Five: self-play in Dota 2; MuZero: 57 Atari games plus Go, chess and shogi without given rules.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 27 authors on the OpenAI Five paper (arXiv 1912.06680); 12 on MuZero.
- Time horizonMulti-year.
- Shelf-life signalsMuZero dropped the need for known rules that AlphaZero had.
- Pipeline roleTier A runs self-play at scale; Tier C studies small reproductions; Tier B and Tier D have no observed role.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Links
R&D TopicSelf-Play Reinforcement Learning with Search
Models
AlphaZeroGoogle DeepMind, reinforcement learningMuZeroGoogle DeepMind, reinforcement learning
OpenAI FiveOpenAI, reinforcement learningAlphaGoGoogle DeepMind, reinforcement learning+1 more
AlphaZeroGoogle DeepMind, reinforcement learningMuZeroGoogle DeepMind, reinforcement learningOpenAI FiveOpenAI, reinforcement learningAlphaGoGoogle DeepMind, reinforcement learning+1 moreUse Casesnone recorded
Who Runs Itnone recorded
Papers
- Student of Games: A unified learning algorithm for both perfect and imperfect information gamesMartin Schmid et al., Science Advances 9, eadg3256 (2023), 2021 state of the art
- Mastering Atari, Go, Chess and Shogi by Planning with a Learned ModelJulian Schrittwieser et al., arXiv preprint, 2019 state of the art
- Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning AlgorithmDavid Silver et al., arXiv preprint, 2017 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/1912.06680 https://arxiv.org/abs/1912.06680
- arxiv.org/abs/1911.08265 https://arxiv.org/abs/1911.08265
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Self-Play Reinforcement Learning with Search", https://aurion.semai.id/topics/self-play-reinforcement-learning-with-search/, records of October 2026.