Model-Free Control — MC, SARSA, Q-learning

Reinforcement Learning — CentraleSupélec · Lecture 3: Model-Free Prediction and Control · Sutton & Barto Examples 6.5 and 6.6

Algorithm (Learn tab)

Exploration ε = Step size α (SARSA, Q-learning) = Monte Carlo step size

α_MC = Random seed

shade: max_a Q(s,a), darker = better arrow: greedy action cliff: −100, back to S the agent its current episode

Reward −1 per step, γ = 1. Keys: → one step, space play/pause.

Sutton & Barto, Reinforcement Learning: An Introduction (2nd ed.), §5.4, §6.4–6.5 · Reinforcement Learning — CentraleSupélec