Sparse tabular multiagent Q-learning

Jelle R. Kok and Nikos Vlassis. Sparse tabular multiagent Q-learning. In Proceedings of the Annual Machine Learning Conference of Belgium and the Netherlands, pp. 65–71, Brussels, Belgium, January 2004.

Download

pdf [163.0kB]  ps.gz [136.8kB]  

Abstract

Multiagent learning problems can in principle be solved by treating the joint actions of the agents as single actions and applying single-agent Q-learning. However, the number of joint actions is exponential in the number of agents, rendering this approach infeasible for most problems. In this paper we investigate a sparse representation of the Q-function by only considering the joint actions in those states in which coordination is actually required. In all other states single-agent Q-learning is applied. This offers a compact state-action value representation, without compromising much in terms of solution quality. We have performed experiments in the predator-prey domain and compared our method to other multiagent reinforcement learning methods with promising results.

BibTeX Entry

@InProceedings{Kok04benelearn,
  author =       {Jelle R. Kok and Nikos Vlassis},
  title =        {Sparse tabular multiagent {Q}-learning},
  address =      {Brussels, Belgium},
  booktitle =    {Proceedings of the Annual Machine Learning
                  Conference of Belgium and the Netherlands},
  year =         {2004},
  pages =        {65--71},
  editor =       {Ann Now\'e, Tom Lenaerts, Kris Steenhaut},
  month =        jan,
  postscript =   {2004/Kok04benelearn.ps.gz},
  pdf =          {2004/Kok04benelearn.pdf},
  abstract =     { Multiagent learning problems can in principle be
                  solved by treating the joint actions of the agents
                  as single actions and applying single-agent
                  Q-learning. However, the number of joint actions is
                  exponential in the number of agents, rendering this
                  approach infeasible for most problems. In this paper
                  we investigate a sparse representation of the
                  Q-function by only considering the joint actions in
                  those states in which coordination is actually
                  required. In all other states single-agent
                  Q-learning is applied. This offers a compact
                  state-action value representation, without
                  compromising much in terms of solution quality. We
                  have performed experiments in the predator-prey
                  domain and compared our method to other multiagent
                  reinforcement learning methods with promising
                  results.}
}

---

Generated by bib2html.pl (written by Patrick Riley) on Tue Oct 31, 2006 19:33:42 UTC