Lecture #17 looked at how state-space search might be considered planning, and how the best route through a state-space might be discovered using reinforcement learning.
Notes are available in two formats:
The lecture covered material form Chapter 21 of the textbook, but covered it in a rather different way from that in the textbook.