Curiosity-driven Exploration by Self-supervised Prediction
https://arxiv.org/abs/1705.05363 Authors: Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, Trevor Darrell Abstract: In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether. In such cases, curiosity can serve as an intrinsic reward signal to enable the agent to explore its environment and learn skills that might be useful later in its life. We formulate curiosity as the error in an agent's ability to predict the consequence of its own actions in a visual feature space learned by a self-supervised inverse dynamics model. Our formulation scales to high-dimensional continuous state spaces like images, bypasses the difficulties of directly predicting pixels, and, critically, ignores the aspects of the environment that cannot affect the agent. The proposed approach is evaluated in two environments: VizDoom and Super Mario Bros. Three broad settings are investigated: 1) sparse extrinsic reward, where curiosity allows for far fewer interactions with the environment to reach the goal; 2) exploration with no extrinsic reward, where curiosity pushes the agent to explore more efficiently; and 3) generalization to unseen scenarios (e.g. new levels of the same game) where the knowledge gained from earlier experience helps the agent explore new places much faster than starting from scratch.
Active Inference and Artificial Curiosity (Distinguished Lecture Series: Karl Friston)
DINO: Emerging Properties in Self-Supervised Vision Transformers (Facebook AI Research Explained)
Reinforcement Learning with sparse rewards
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
Yann LeCun: Self-Supervised Learning Explained | Lex Fridman Podcast Clips
The Strange Math That Predicts (Almost) Anything
ReBeL - Combining Deep Reinforcement Learning and Search for Imperfect-Information Games (Explained)
What Is Self-Supervised Learning and Why Care?
An introduction to Policy Gradient methods - Deep Reinforcement Learning
Training Sand to Think: Artificial General Intelligence & Future of Physics
Model Based RL Finally Works!
Training AI Without Writing A Reward Function, with Reward Modelling
Paper Club with Peter - Maximum Entropy Inverse Reinforcement Learning
Can a Reinforcement Learning Agent Learn with NO Rewards? Intrinsic Curiosity Coding Tutorial
Train Your Brain to Never Forget (5 Feynman Habits)
Yann LeCun - Self-Supervised Learning: The Dark Matter of Intelligence (FAIR Blog Post Explained)
Representation-driven Option Discovery in Reinforcement Learning, Marlos C. Machado
Juergen Schmidhuber: The Algorithmic Principle Behind Curiosity and Creativity
BYOL: Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (Paper Explained)
