Reevaluating Policy Gradient Methods for Imperfect-Information Games
Max Rudolph, Nathan Lichtlé, Sobhan Mohammadpour, Alexandre Bayen, J. Zico Kolter, Amy Zhang, Gabriele Farina, Eugene Vinitsky, Samuel Sokota · 2025
Read Reevaluating Policy Gradient Methods for Imperfect-Information Games by Max Rudolph, Nathan Lichtlé, Sobhan Mohammadpour, Alexandre Bayen, J. Zico Kolter, Amy Zhang, Gabriele Farina, Eugene Vinitsky, Samuel Sokota (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08938.txt.
Description
Max Rudolph and co-authors retest a settled belief in game AI: that plain policy gradient methods fail in adversarial games with hidden information, so special methods are needed. They built exact exploitability measures for five large games, a first for broad access, and ran the largest comparison yet in this area, over seven thousand training runs.
Methods based on fictitious play, double oracle and counterfactual regret did not beat generic policy gradient methods such as PPO. The result was prompted by recent gains from a mirror descent method. Code for the benchmark and the exploitability tools was released in public, so others can rerun and extend the contest.
Catalog record
| CREATOR | Max Rudolph, Nathan Lichtlé, Sobhan Mohammadpour, Alexandre Bayen, J. Zico Kolter, Amy Zhang, Gabriele Farina, Eugene Vinitsky, Samuel Sokota |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-08938