Questions
- Form
- What’s the advantage of stochastic policies over deterministic policies, especially when stochastic policies
- RL is an evolutionary algorithm
Literature Review
More papers
- DiscoRL: Discovering state-of-the-art reinforcement learning algorithms | Nature: David silver last work at deepmind
- MuZero: Schrittwieser, Julian, et al. “Mastering atari, go, chess and shogi by planning with a learned model.” Nature 588.7839 (2020): 604-609.
- A3C: [1602.01783] Asynchronous Methods for Deep Reinforcement Learning
- [2211.07819] General Intelligence Requires Rethinking Exploration
- Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries
- Reinforcement Learning for LLMs: The Complete Guide
- GitHub - TuragaLab/flybody: MuJoCo fruit fly body model and locomotion RL tasks · GitHub
Q-Learning
- DQN: Human-level control through deep reinforcement learning | Nature
- [2010.14498] Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning – Argues methods used for data-efficient RL like off-policy online RL and offline RL (on the extreme case) exhibits under-parametrisation with more training iteration. They show the loss of expressivity due to drop in the rank of the learned value network features, and correlates that with the performance drop usually observed in bootstrapped DeepRL algorithms like Fitted Q-Iteration, TD learning.
- How to mitigate it?
Policy Gradient
- Silver, David, et al. “Deterministic policy gradient algorithms.” Pmlr, 2014.
- DDPG – Lillicrap, Timothy P., et al. “Continuous control with deep reinforcement learning.” arXiv preprint arXiv:1509.02971 (2015).
- TD3 – Fujimoto, Scott, Herke Hoof, and David Meger. “Addressing function approximation error in actor-critic methods.” International conference on machine learning. Pmlr, 2018.
- PPO: [1707.06347] Proximal Policy Optimization Algorithms
- SAC: [1812.05905] Soft Actor-Critic Algorithms and Applications
Distillation
- [2306.13649] On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- On-Policy Distillation - Thinking Machines Lab
- Reinforcement Learning via Self-Distillation | alphaXiv
- [1910.01108] DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- [2303.17651] Self-Refine: Iterative Refinement with Self-Feedback
- [2106.01345] Decision Transformer: Reinforcement Learning via Sequence Modeling
- [2305.20050] Let’s Verify Step by Step
Inverse RL:
RL x Diffusion
- Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Development | alphaXiv
- Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization | alphaXiv
- [2603.27450] FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policies
- [2208.06193] Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning
Environments
- OpenReward: Train and evaluate agents on 380+ reinforcement learning environments with zero infrastructure setup.
- https://puffer.ai/
- Pavlov’s List: RL envs and data startups
Repositories
- GitHub - PufferAI/PufferLib: Puffing up reinforcement learning · GitHub
- GitHub - Al-does/RL-Harness: RL Harness — local RLlib harness for custom RL experiments · GitHub, GitHub - Al-does/alex-rl-experiments: Alex’s personal RL experiment recipes for rl-harness · GitHub
- GitHub - ray-project/ray: Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. · GitHub
Course
- RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning - YouTube
- CS234: Reinforcement Learning Winter 2026
- CS 224R Deep Reinforcement Learning | Stanford
- CS 185/285 | UC Berkeley
- CS885 Winter 2025 - Schedule | UWaterloo
- CS 2824 Foundations of RL | Harvard
- Home | RL Theory
Tutorials