Empirically Evaluating Multiagent Reinforcement Learning Algorithms
By Asher Lipson
Abstract:
This talk will present my MSc research. The work that will be described is
the development of a platform for running experiments on multiagent
reinforcement learning algorithms and an empirical evaluation that was conducted
on the platform. The setting under consideration is game theoretic in which a
single normal form game is repeatedly played.
There has been a large body of work focusing on introducing new algorithms to
achieve certain goals such as guaranteeing values in a game, converging to a
Nash equilibrium or minimizing total regret.
Currently, we have an understanding of how some of these algorithms work in
limited settings, but lack a broader understanding of which algorithms perform
well against each other and how they perform on a larger variety of games.
We describe our development of a platform that allows large scale tests to be
run, where multiple algorithms are played against one another on a variety of
games. We also present the results of an empirical test that to our knowledge
includes the largest combination of game instances and algorithms used in the
multiagent learning literature. To demonstrate the usefulness of the platform,
we provide evidence for a number of claims and hypotheses. This includes claims
related to convergence to a Nash equilibrium, reward, regret and best response
metrics and claims dealing with estimating an opponent's strategy.
The two major contributions of this work are a software platform for running
large experimental tests and empirical results that provide insight into the
performance of various algorithms.