Uncertainty-Aware Action Advising for Deep Reinforcement Learning Agents | AISC
19For slides and more information on the paper, visit https://ai.science/e/uncertainty-aware-action-advising-for-deep-reinforcement-learning-agents--gvft1hbacsycJX0VCfht
Speaker: Matthew Taylor, Felipe Leno da Silva; Discussion Facilitator: Susan Shu Chang
Motivation:
Although Reinforcement Learning (RL) has been one of the most successful approaches for learning in sequential decision making problems, the sample-complexity of RL techniques still represents a major challenge for practical applications. To combat this challenge, whenever a competent policy (e.g., either a legacy system or a human demonstrator) is available, the agent could leverage samples from this policy (advice) to improve sample-efficiency.
In this work, we propose Requesting Confidence-Moderated Policy advice (RCMP), an action-advising framework where the agent asks for advice when its epistemic uncertainty is high for a certain state. RCMP takes into account that the advice is limited and might be suboptimal. Our empirical evaluations show that RCMP performs better than Importance Advising, not receiving advice, and receiving it at random states in Gridworld and Atari Pong scenarios.