LEAP

From PRLT
Jump to: navigation, search


This project aims at analyzing the effects of function approximation techniques on Reinforcement Learning algorithms from a theoretical and experimental point of view.

Introduction

Reinforcement Learning algorithms suffer from the curse of dimensionality and it is often unfeasible to deal with problems characterized by huge state and action spaces. Therefore, function approximation techniques (e.g., state aggregation, neural networks and so forth) are usually applied to learn an accurate approximation of the (action) value function with few parameters. Although function approximation has been widely applied to RL problems, many theoretical and practical issues are still open.

This research project aims at the analysis of theoretical issues such as convergence and optimality and at the development of novel algorithms that incrementally adapt their structure according to the characteristics of the problem at hand.

Ongoing work

We are investigating the following topics the dynamical adaptation of the state representation. The LEAP (Learning Entities Adaptive Partitioning) algorithm refines an initial state representation until the target values received from the environment are coherent with respect to the action value function computed so far. Unlike other similar algorithms, LEAP is based on linear averager function approximation and does not limit to refinment of a coarse aggregation but is able to aggregate states when a too fine state representation is usless for learning a nearly-optimal policy.

Resources

People involved

  • Andrea Bonarini, Full Professor
  • Alessandro Lazaric, PhD Student
  • Marcello Restelli, PhD

Publications

Andrea Bonarini, Alessandro Lazaric, Marcello Restelli - Learning in Complex Environments through Multiple Adaptive Partitions
Workshop on Planning, Learning and Monitoring with Uncertainty and Dynamic Worlds, ECAI , 2006
Bibtex
Auteur : Andrea Bonarini, Alessandro Lazaric, Marcello Restelli
Titre : Learning in Complex Environments through Multiple Adaptive Partitions
Dans : Workshop on Planning, Learning and Monitoring with Uncertainty and Dynamic Worlds, ECAI -
Adresse :
Date : 2006

Andrea Bonarini, Alessandro Lazaric, Marcello Restelli - LEAP: Learning Entities Adaptive Partitioning
Reinforcement Learning Benchmarks and Bake-offs II, NIPS 2005 , 2005
Bibtex
Auteur : Andrea Bonarini, Alessandro Lazaric, Marcello Restelli
Titre : LEAP: Learning Entities Adaptive Partitioning
Dans : Reinforcement Learning Benchmarks and Bake-offs II, NIPS 2005 -
Adresse :
Date : 2005