1sec.ai
Back to feed
research1d ago

A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise

aarXivscore 0.24

A stochastic differential equation (SDE) approximation for linear TD(0) under Markovian noise is introduced, capturing the contraction dynamics and providing a more accurate description of the error floor than the classical ODE.

Key takeaways

  • Introduced a stochastic differential equation (SDE) approximation for linear TD(0) under Markovian noise.
  • The SDE model captures the contraction dynamics governed by the projected Bellman operator.
  • The SDE approximation provides a more accurate description of the error floor than the classical ODE.
#td-learning#stochastic-diffusion#markovian-noise
Read the original
Feed
research1d ago

A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise

aarXiv

A stochastic differential equation (SDE) approximation for linear TD(0) under Markovian noise is introduced, capturing the contraction dynamics and providing a more accurate description of the error floor than the classical ODE.

Key takeaways

  • Introduced a stochastic differential equation (SDE) approximation for linear TD(0) under Markovian noise.
  • The SDE model captures the contraction dynamics governed by the projected Bellman operator.
  • The SDE approximation provides a more accurate description of the error floor than the classical ODE.
#td-learning#stochastic-diffusion#markovian-noise
Read at arXiv