research1d
A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise
A stochastic differential equation (SDE) approximation for linear TD(0) under Markovian noise is introduced, capturing the contraction dynamics and providing a more accurate description of the error floor than the classical ODE.
Key takeaways
- Introduced a stochastic differential equation (SDE) approximation for linear TD(0) under Markovian noise.
- The SDE model captures the contraction dynamics governed by the projected Bellman operator.
- The SDE approximation provides a more accurate description of the error floor than the classical ODE.