Sparse cooperative Q-learning

Jelle R. Kok and Nikos Vlassis. Sparse cooperative Q-learning. In Proceedings of the International Conference on Machine Learning, pp. 481–488, ACM, Banff, Canada, July 2004.

Download

pdf [150.8kB]  ps.gz [156.3kB]  

Abstract

Learning in multiagent systems suffers from the fact that both the state and the action space scale exponentially with the number of agents. In this paper we are interested in using Q-learning to learn the coordinated actions of a group of cooperative agents, using a sparse representation of the joint state-action space of the agents. We first examine a compact representation in which the agents need to explicitly coordinate their actions only in a predefined set of states. Next, we use a coordination-graph approach in which we represent the Q-values by value rules that specify the coordination dependencies of the agents at particular states. We show how Q-learning can be efficiently applied to learn a coordinated policy for the agents in the above framework. We demonstrate the proposed method on the predator-prey domain, and we compare it with other related multiagent Q-learning methods.

BibTeX Entry

@InProceedings{Kok04icml,
  author =       {Jelle R. Kok and Nikos Vlassis},
  title =        {Sparse cooperative {Q}-learning},
  address =      {Banff, Canada},
  booktitle =    {Proceedings of the International Conference on
                  Machine Learning},
  year =         2004,
  month =        jul,
  pages =        {481--488},
  editor =       {Russ Greiner and Dale Schuurmans},
  publisher =    {ACM},
  postscript =   {2004/Kok04icml.ps.gz},
  pdf =          {2004/Kok04icml.pdf},
  abstract =     { Learning in multiagent systems suffers from the
                  fact that both the state and the action space scale
                  exponentially with the number of agents. In this
                  paper we are interested in using Q-learning to learn
                  the coordinated actions of a group of cooperative
                  agents, using a sparse representation of the joint
                  state-action space of the agents. We first examine a
                  compact representation in which the agents need to
                  explicitly coordinate their actions only in a
                  predefined set of states. Next, we use a
                  coordination-graph approach in which we represent
                  the Q-values by value rules that specify the
                  coordination dependencies of the agents at
                  particular states. We show how Q-learning can be
                  efficiently applied to learn a coordinated policy
                  for the agents in the above framework. We
                  demonstrate the proposed method on the predator-prey
                  domain, and we compare it with other related
                  multiagent Q-learning methods. }
}

---

Generated by bib2html.pl (written by Patrick Riley) on Tue Oct 31, 2006 19:33:42 UTC