Explorations in Efficient Reinforcement Learning

Abstract

This thesis describes reinforcement learning (RL) methods which can solve sequential decision making problems by learning from trial and error. Sequential decision making problems are problems in which an artificial agent interacts with a specific environment through its sensors (to get inputs) and effectors (to make actions). To measure the goodness of some agent's behavior, a reward function is used which determines how much an agent is rewarded or penalized for performing particular actions in particular environmental states. The goal is to find an action selection policy for the agent which maximizes the cumulative reward collected in the future. In RL, an agent's policy maps sensorbased inputs to actions. To evaluate a policy, a value function is learned which returns for each possible state the future cumulative reward collected by following the current policy. Given a value function, we can simply select the action with the largest value. In order to learn a value function for a specific problem, reinforcement learning methods simulate a policy and use the resulting agent's experiences consisting of <state,action,reward,next-state> quadruples. There are different RL problems and different RL methods for solving them. We describe different categories of problems and introduce new methods for solving them.