Publications

2026

1. NeurIPS’26

   Independent Learning of Nash Equilibria in Partially Observable Markov Potential Games with Decoupled Dynamics

   Philip Jordan and Maryam Kamgarpour

   In Advances in Neural Information Processing Systems 39, 2026

   Abstract Bib PDF Poster

   We study Nash equilibrium learning in partially observable Markov games (POMGs), a multi-agent reinforcement learning framework in which agents cannot fully observe the underlying state. Prior work in this setting relies on centralization or information sharing, and suffers from sample and computational complexity that scales exponentially in the number of players. We focus on a subclass of POMGs with independent state transitions, where agents remain coupled through their rewards, and assume that the underlying fully observed Markov game is a Markov potential game. For this class, we present an independent learning algorithm in which players, observing only their own actions and observations and without communication, jointly converge to an approximate Nash equilibrium. Due to partial observability, optimal policies may in general depend on the full action-observation history. Under a filter stability assumption, we show that policies based on finite history windows provide sufficient approximation guarantees. This enables us to approximate the POMG by a surrogate Markov game that is near-potential, leading to quasi-polynomial sample and computational complexity for independent Nash equilibrium learning in the underlying POMG.

   @inproceedings{jordan2026independent,
     title = {Independent Learning of Nash Equilibria in Partially Observable Markov Potential Games with Decoupled Dynamics},
     author = {Jordan, Philip and Kamgarpour, Maryam},
     booktitle = {Advances in Neural Information Processing Systems 39},
     year = {2026},
   }

2. CDC’26

   Model-Based Learning of Near-Optimal Finite-Window Policies in POMDPs

   Philip Jordan and Maryam Kamgarpour

   In IEEE 65th Conference on Decision and Control, 2026

   Abstract Bib PDF

   We study model-based learning of finite-window policies in tabular partially observable Markov decision processes (POMDPs). A common approach to learning under partial observability is to approximate unbounded history dependencies using finite action-observation windows. This induces a finite-state Markov decision process (MDP) over histories, referred to as the superstate MDP. Once a model of this superstate MDP is available, standard MDP algorithms can be used to compute optimal policies, motivating the need for sample-efficient model estimation. Estimating the superstate MDP model is challenging because trajectories are generated by interaction with the original POMDP, creating a mismatch between the sampling process and target model. We propose a model estimation procedure for tabular POMDPs and analyze its sample complexity. Our analysis exploits a connection between filter stability and concentration inequalities for weakly dependent random variables. As a result, we obtain tight sample complexity guarantees for estimating the superstate MDP model from a single trajectory. Combined with value iteration, this yields approximately optimal finite-window policies for the POMDP.

   @inproceedings{jordan2026model,
     title = {Model-Based Learning of Near-Optimal Finite-Window Policies in POMDPs},
     author = {Jordan, Philip and Kamgarpour, Maryam},
     booktitle = {IEEE 65th Conference on Decision and Control},
     organization = {IEEE},
     year = {2026},
   }

3. ICML’26

   Oral

   Nash Equilibria in Games with Playerwise Concave Coupling Constraints: Existence and Computation

   Philip Jordan and Maryam Kamgarpour

   In Proceedings of the 43rd International Conference on Machine Learning, 2026

   Selected for oral presentation

   Abstract Bib PDF Code Poster Slides

   Selected for oral presentation

   We study the existence and computation of Nash equilibria in concave games where the players’ admissible strategies are subject to shared coupling constraints. Under playerwise concavity of constraints, we prove existence of Nash equilibria. Our proof leverages topological fixed point theory and novel structural insights into the contractibility of feasible sets, and relaxes strong assumptions for existence in prior work. Having established existence, we address the question of whether in the presence of coupling constraints, playerwise independent learning dynamics have convergence guarantees. We address this positively for the class of potential games by designing a convergent algorithm. To account for the possibly nonconvex feasible region, we employ a log barrier regularized gradient ascent with adaptive stepsizes. Starting from an initial feasible strategy profile and under exact gradient feedback, the proposed method converges to an ε-approximate constrained Nash equilibrium within O(ε^-3) iterations.

   @inproceedings{jordan2026nash,
     title = {Nash Equilibria in Games with Playerwise Concave Coupling Constraints: Existence and Computation},
     author = {Jordan, Philip and Kamgarpour, Maryam},
     booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
     year = {2026},
   }

2024

1. AISTATS’24

   Independent Learning in Constrained Markov Potential Games

   Philip Jordan, Anas Barakat, and Niao He

   In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, 2024

   Abstract Bib PDF Code Poster Slides

   Constrained Markov games offer a formal mathematical framework for modeling multi-agent reinforcement learning problems where the behavior of the agents is subject to constraints. In this work, we focus on the recently introduced class of constrained Markov Potential Games. While centralized algorithms have been proposed for solving such constrained games, the design of converging independent learning algorithms tailored for the constrained setting remains an open question. We propose an independent policy gradient algorithm for learning approximate constrained Nash equilibria: Each agent observes their own actions and rewards, along with a shared state. Inspired by the optimization literature, our algorithm performs proximal-point-like updates augmented with a regularized constraint set. Each proximal step is solved inexactly using a stochastic switching gradient algorithm. Notably, our algorithm can be implemented independently without a centralized coordination mechanism requiring turn-based agent updates. Under some technical constraint qualification conditions, we establish convergence guarantees towards constrained approximate Nash equilibria. We perform simulations to illustrate our results.

   @inproceedings{jordan2024independent,
     title = {Independent Learning in Constrained {M}arkov Potential Games},
     author = {Jordan, Philip and Barakat, Anas and He, Niao},
     booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics},
     pages = {4024--4032},
     year = {2024},
     editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen},
     volume = {238},
     series = {Proceedings of Machine Learning Research},
     publisher = {PMLR},
     url = {https://proceedings.mlr.press/v238/jordan24a.html},
   }

2. AAMAS’24

   Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence

   Philip Jordan, Florian Grötschla, Flint Xiaofeng Fan, and 1 more author

   In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2024

   Abstract Bib PDF Code Poster Slides

   In Federated Reinforcement Learning (FRL), agents aim to collaboratively learn a common task, while each agent is acting in its local environment without exchanging raw trajectories. Existing approaches for FRL either (a) do not provide any fault-tolerance guarantees (against misbehaving agents), or (b) rely on a trusted central agent (a single point of failure) for aggregating updates. We provide the first decentralized Byzantine fault-tolerant FRL method. Towards this end, we first propose a new centralized Byzantine fault-tolerant policy gradient (PG) algorithm that improves over existing methods by relying only on assumptions standard for non-fault-tolerant PG. Then, as our main contribution, we show how a combination of robust aggregation and Byzantine-resilient agreement methods can be leveraged in order to eliminate the need for a trusted central entity. Since our results represent the first sample complexity analysis for Byzantine fault-tolerant decentralized federated non-convex optimization, our technical contributions may be of independent interest. Finally, we corroborate our theoretical results experimentally for common RL environments, demonstrating the speed-up of decentralized federations w.r.t. the number of participating agents and resilience against various Byzantine attacks.

   @inproceedings{jordan2024decentralized,
     author = {Jordan, Philip and Gr\"{o}tschla, Florian and Fan, Flint Xiaofeng and Wattenhofer, Roger},
     title = {Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence},
     year = {2024},
     isbn = {9798400704864},
     publisher = {International Foundation for Autonomous Agents and Multiagent Systems},
     address = {Richland, SC},
     booktitle = {Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems},
     pages = {964–972},
     numpages = {9},
     keywords = {byzantine fault-tolerance, decentralization, federated reinforcement learning, policy gradient, robust optimization},
     series = {AAMAS '24},
   }