Sparse cooperative Q-learning: extended abstract

Jelle R. Kok and Nikos Vlassis. Sparse cooperative Q-learning: extended abstract. In Proceedings of the 16th Belgian-Dutch Conference on Artifical Intelligence, pp. 361–362, Groningen, The Netherlands, October 2004.

Download

pdf [55.3kB]  ps.gz [61.3kB]  

Abstract

Learning in multiagent systems suffers from the fact that both the state and the action space scale exponentially with the number of agents. In this paper we are interested in using Q-learning to learn the coordinated actions of a group of cooperative agents, using a sparse representation of the joint state-action space of the agents. We first examine a compact representation in which the agents need to explicitly coordinate their actions only in a predefined set of states. Next, we use a coordination-graph approach in which we represent the Q-values by value rules that specify the coordination dependencies of the agents at particular states. We show how Q-learning can be efficiently applied to learn a coordinated policy for the agents in the above framework. We demonstrate the proposed method on the predator-prey domain, and we compare it with other related multiagent Q-learning methods.

BibTeX Entry

@InProceedings{Kok04bnaic,
  author =       {Jelle R. Kok and Nikos Vlassis},
  title =        {Sparse cooperative {Q}-learning: extended
                  abstract},
  address =      {Groningen, The Netherlands},
  booktitle =    {Proceedings of the 16th Belgian-Dutch Conference on
                  Artifical Intelligence},
  year =         2004,
  month =        oct,
  pages =        {361--362},
  editor =       {Rineke Verbrugge and Niels Taatgen and Lambert
                  Schomaker},
  postscript =   {2004/Kok04bnaic.ps.gz},
  pdf =          {2004/Kok04bnaic.pdf},
  abstract =     { Learning in multiagent systems suffers from the
                  fact that both the state and the action space scale
                  exponentially with the number of agents. In this
                  paper we are interested in using Q-learning to learn
                  the coordinated actions of a group of cooperative
                  agents, using a sparse representation of the joint
                  state-action space of the agents. We first examine a
                  compact representation in which the agents need to
                  explicitly coordinate their actions only in a
                  predefined set of states. Next, we use a
                  coordination-graph approach in which we represent
                  the Q-values by value rules that specify the
                  coordination dependencies of the agents at
                  particular states. We show how Q-learning can be
                  efficiently applied to learn a coordinated policy
                  for the agents in the above framework. We
                  demonstrate the proposed method on the predator-prey
                  domain, and we compare it with other related
                  multiagent Q-learning methods. }
}

---

Generated by bib2html.pl (written by Patrick Riley) on Tue Oct 31, 2006 19:33:42 UTC