Empirically Evaluating Multiagent Reinforcement Learning Algorithms

By Asher Lipson


This talk will present my MSc research. The work that will be described is the development of a platform for running experiments on multiagent reinforcement learning algorithms and an empirical evaluation that was conducted on the platform. The setting under consideration is game theoretic in which a single normal form game is repeatedly played.

There has been a large body of work focusing on introducing new algorithms to achieve certain goals such as guaranteeing values in a game, converging to a Nash equilibrium or minimizing total regret.
Currently, we have an understanding of how some of these algorithms work in limited settings, but lack a broader understanding of which algorithms perform well against each other and how they perform on a larger variety of games.

We describe our development of a platform that allows large scale tests to be run, where multiple algorithms are played against one another on a variety of games. We also present the results of an empirical test that to our knowledge includes the largest combination of game instances and algorithms used in the multiagent learning literature. To demonstrate the usefulness of the platform, we provide evidence for a number of claims and hypotheses. This includes claims related to convergence to a Nash equilibrium, reward, regret and best response metrics and claims dealing with estimating an opponent's strategy.

The two major contributions of this work are a software platform for running large experimental tests and empirical results that provide insight into the performance of various algorithms.

Back to the LCI Forum page