Deep Reinforcement Learning Algorithms for Control
General algorithms that learn control policies from reward, online or from fixed logged data.
Tier Grades
Tier A: Frontier AI labs
feasible
OpenAI and Google DeepMind publish the core algorithms and scale IMPALA to thousands of machines.
Tier B: Indonesian AI labs
feasible
PPO and SAC run on simulated robotics benchmarks with modest compute.
Tier C: Institution units such as ITB
feasible
PPO and SAC are standard teaching and research baselines in university labs.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Offline RL with CQL learns from logged data without interaction, which fits a process-optimisation proof of concept.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeIMPALA scales to thousands of machines; PPO allows multiple epochs of minibatch updates per sample; compute per task not stated.
- DataIMPALA: DMLab-30 and Atari-57; CQL: fixed offline datasets without further interaction.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 5 authors on the PPO paper (arXiv 1707.06347); 12 on IMPALA.
- Time horizonYears.
- Shelf-life signalsPPO remains the default policy-gradient method years after 2017; offline RL (CQL) grew for settings where interaction is costly.
- Pipeline roleTier A publishes algorithms; Tier B and Tier C apply them in simulation; Tier D uses offline RL on client logs.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
CQL (Conservative Q-Learning)reinforcement learningexemplarSAC (Soft Actor-Critic)reinforcement learningexemplarPPO (Proximal Policy Optimization)reinforcement learningexemplar
π*0.6robot policyused by
Agent57reinforcement learningproduced by
TD3 (Twin Delayed DDPG)reinforcement learningproduced byIMPALAreinforcement learningproduced byDQN (Deep Q-Network)reinforcement learningproduced byLinks
R&D TopicDeep Reinforcement Learning Algorithms for Control
ModelsPPO (Proximal Policy Optimization)reinforcement learningSAC (Soft Actor-Critic)UC Berkeley, reinforcement learningCQL (Conservative Q-Learning)UC Berkeley, reinforcement learningπ*0.6Physical Intelligence, robot policy+4 more
SAC (Soft Actor-Critic)UC Berkeley, reinforcement learningCQL (Conservative Q-Learning)UC Berkeley, reinforcement learningπ*0.6Physical Intelligence, robot policy+4 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 8
π*0.6Physical Intelligence, robot policyused byCQL (Conservative Q-Learning)UC Berkeley, Google, reinforcement learningexemplarAgent57Google DeepMind, reinforcement learningproduced byDQN (Deep Q-Network)Google DeepMind, reinforcement learningproduced byIMPALAGoogle DeepMind, reinforcement learningproduced bySAC (Soft Actor-Critic)UC Berkeley, reinforcement learningexemplarTD3 (Twin Delayed DDPG)McGill University, reinforcement learningproduced byPPO (Proximal Policy Optimization)reinforcement learningexemplarPapers
- Playing Atari with Deep Reinforcement LearningVolodymyr Mnih et al., arXiv preprint, 2013 founding
- Continuous control with deep reinforcement learningTimothy P. Lillicrap et al., arXiv preprint, 2015 founding
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic ActorTuomas Haarnoja et al., ICML 2018 (per arXiv comment), 2018 state of the art
- Proximal Policy Optimization AlgorithmsJohn Schulman et al., arXiv preprint, 2017 state of the art
- Deep Reinforcement Learning: An OverviewYuxi Li, arXiv preprint, 2017 survey
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/1707.06347 https://arxiv.org/abs/1707.06347
- arxiv.org/abs/1801.01290 https://arxiv.org/abs/1801.01290
- arxiv.org/abs/2006.04779 https://arxiv.org/abs/2006.04779
- arxiv.org/abs/1802.01561 https://arxiv.org/abs/1802.01561
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Deep Reinforcement Learning Algorithms for Control", https://aurion.semai.id/topics/deep-reinforcement-learning-for-control/, records of October 2026.