New Zealand


The recent mean field game (MFG) formalism has enabled the application of inverse reinforcement learning (IRL) methods in large-scale multi-agent systems, with the goal of inferring reward signals that can explain demonstrated behaviours of large populations. The existing IRL methods for MFGs are built upon reducing an MFG to a Markov decision process (MDP) defined on the collective behaviours and average rewards of the population. However, this paper reveals that the reduction from MFG to MDP holds only for the fully cooperative setting. This flaw invalidates existing IRL methods on MFGs with non-cooperative environments. To measure more general behaviours in large populations, we study the use of individual behaviours to directly infer ground-truth reward functions for MFGs. We propose Mean Field IRL (MFIRL), the first IRL framework for MFGs that can handle both cooperative and non-cooperative environments. Based on this theoretically justified framework, we develop a practical algorithm effective for MFGs with continuous state-action spaces and unknown dynamics. We evaluate MFIRL on both cooperative and mixed cooperative-competitive scenarios with many agents. Results show that MFIRL excels in reward recovery, sample efficiency and robustness to changing dynamics, compared to the method based on the reduction to MDP.

AAMAS 2022

Individual-Level Inverse Reinforcement Learning for Mean Field Games

mean field nash equilibrium

individual level

mean field games

inverse reinforcement learning

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

technical paper

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).

A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).


Evolutionary Game Theory (EGT) studies evolving populations of agents that interact through normal form games whose outcome determines each individual's evolutionary fitness. In many applications, EGT models are extended to include spatial effects in which the agents are located in a structured population such as a graph. We propose a Mean Field Game (MFG) generalization, denoted Pair-MFG, of the spatial evolutionary game model such that the behavior of a given spatial evolutionary game (or more specifically the behavior of its pair approximation) is a special case trajectory of the corresponding MFG. The proposed Pair-MFG model also allows for the formulation of the spatial evolutionary game as a control problem, opening up additional avenues of research into controlling the outcomes of these games. 

The state evolution equations of the proposed model are highly nonlinear and none of the equations in the system are necessarily convex. This necessitates different numerical methods as compared to those for traditional Linear Quadratic Gaussian MFGs. We provide a method for solving this new Pair-MFG model using fixed point iteration with time-dependent proximal terms and show empirically that this method is capable of finding a solution to a selection of EGT games.

A Mean Field Game Model of Spatial Evolutionary Games


Emergent communication has made strides towards learning communication from scratch, but has thus far only focused on protocols that resemble human language. In nature, multi-agent cooperation gives rise to a wide range of communication that varies in structure and complexity. In this work, we recognize the full spectrum of communication that exists in nature and propose studying lower-level communication. Specifically, we study emergent implicit communication in the context of decentralized multi-agent learning in difficult, sparse reward environments. However, learning to coordinate in such environments is challenging. To this end, we propose a curriculum-driven multi-agent strategy that combines: (i) velocity-based environment shaping, tailored to the skill level of the multi-agent team; and (ii) a behavioral curriculum that helps agents learn successful single-agent behaviors as a precursor to learning multi-agent behaviors. Pursuit-evasion experiments show that our approach learns effective coordination, significantly outperforming sophisticated analytical and learned policies. The policies learned by our method complete the pursuit-evasion task when pursuers move at only one-half of the evader's speed, whereas the highest-performing baseline fails at $80\%$ of the evader's speed. Moreover, we examine the use of implicit signals in coordination through position-based social influence. We show that pursuers trained with our strategy exchange more than twice as much information (in bits) than baseline methods, indicating that our method has learned, and relies heavily on, the exchange of implicit signals.

Multi-Agent Curricula and Emergent Implicit Signaling


We address the scaling of equilibrium computation in Mean Field Games (MFGs) by using Online Mirror Descent (OMD). We show that continuous-time OMD provably converges to a Nash equilibrium under a natural and well-motivated set of monotonicity assumptions. A thorough experimental investigation on various single, and multi-population, MFGs shows that OMD outperforms traditional algorithms such as Fictitious Play (FP). We empirically show that OMD scales and converges significantly faster than FP by solving, for the first time to our knowledge, examples of MFGs with hundreds of billions states. As such, this study establishes the state-of-the-art for learning in large-scale multi-agent and multi-population games.

Scaling Mean Field Games with Online Mirror Descent


During the COVID-19 pandemic, governments have struggled to devise strategies to slow down the spread of the virus. This struggle happens because pandemics are complex scenarios with many unknown variables. In this context, simulated models are used to evaluate strategies for mitigating this and future pandemics. In this paper, we propose a simulator that analyses small communities by using real geographical data to model the road interactions and the agent's behaviors. Our simulator consists of three different modules: Environment, Mobility, and Infection module. The environment module recreates an area based on map data, including houses, restaurants, and roads. The mobility module drives the agents to move in the map based on their work schedule and needs, such as eating at restaurants, doing groceries, and going to work. The infection module simulates four cases of infection: on the road, at home, at a building, and off the map. We simulate the surrounding areas of a university campus. We design four intervention strategies and compare them to a scenario without any intervention. The interventions are: 1) PCR testing and self-isolation if positive; 2) reducing working hours at barbershops and restaurants; 3) closing barbershops and restaurants; 4) closing grocery stores and restaurants and providing delivery instead. For all scenarios, we observe the two types of areas where most infection happens: hubs, where people from different occupations can meet (e.g., restaurants); and non-hubs, where people with the same occupation meet (e.g., offices). The simulations show that all interventions reduce the total number of infected agents by a large margin. We observed that interventions targeting hubs (2-4) did not impact the infection at non-hubs. In addition, the intervention targeting people's behavior (1) ended up creating a cluster at the testing center.

Using Agent-Based Simulator to Assess Interventions Against COVID-19 in a Small Community Generated from Map Data


In response to COVID-19, many countries have mandated social distancing and banned large group gatherings in order to slow down the spread of SARS-CoV-2. Despite the effectiveness of these mandates, the only way to eradicate the disease is to achieve herd immunity through vaccination. In order to increase the accessibility of vaccines, states such as Virginia have funded mobile vaccination centers to distribute vaccines across the state. When choosing where to place these sites, there are two important factors to take into account: accessibility and equity. We develop efficient algorithms with theoretical guarantees on both of these aspects. Finally, we run computational experiments on real world data to show the efficacy of our methods.

Deploying Vaccine Distribution Sites for Improved Accessibility and Equity to Support Pandemic Response


Classification and clustering are prevalent to recognize the identity and the community of nodes in graphs. Several studies analyzed graph structure to build graph neural network (GNN) models for reducing the accuracy of node classification and clustering recently. The existing defense usually modified the structure of models and adopted countermeasure training to enhance the robustness of the node classification and clustering. However, these defense works are model-oriented and not robust. To alleviate the problem, this paper first proposed a robust node classification metric based on residual entropy. More concrete, the accuracy of node classification is proven robustly by maximizing the residual entropy of the nodes theoretically. Then two graph generative algorithms are proposed to resist against two kinds of GNN-based attacks, the untargeted attack and the targeted attack, respectively. Finally, experimental analysis show that the proposed algorithms outperform the existing defense works under five classic datasets.

Residual Entropy-based Graph Generative Algorithms


Agent-based models (ABMs) highlight the importance of simulation validation, such as qualitative face validation and quantitative empirical validation. In particular, we focused on quantitative validation by adjusting simulation input parameters of the ABM. This study introduces an automatic calibration framework that combines the suggested dynamic and heterogeneous calibration methods. Specifically, the dynamic calibration fits the simulation results to the real-world data by automatically capturing suitable simulation time to adjust the simulation parameters, which continuously regulates the simulation diverging from the real world. Conversely, the heterogeneous calibration fits the distributional discrepancy between individuals in simulation and the real world by adjusting agent related parameters cluster-wisely.

Automatic calibration framework of agent-based models for dynamic and heterogeneous parameters


We present a logic of responsibility. Extending stit theory with epistemic, doxastic, deontic, and intentional modalities, we provide logic-based characterizations of several modes of responsibility.

A Stit Logic of Responsibility

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1607.pdf

**Abstract:**



A Graph Neural Network Reasoner for Game Description Language

Existing approaches to coalition formation often assume that requirements associated with tasks are precisely specified by the human operator. However, prior work has demonstrated that humans, while extremely adept at solving complex problems, struggle to explicitly state their solution strategy. Further, existing approaches often ignore the fact that experts may utilize different, but equally-valid, solutions (i.e., heterogeneous strategies} to the same problem. In this work, we propose a two-part framework to address these challenges. First, we tackle the challenge of inferring implicit strategies directly from expert demonstrations of coalition formation. To this end, we model and infer such heterogeneous strategies as capability-based requirements associated with each task. Next, we propose a method capable of adaptively selecting one of the inferred strategies that best suits the target team without requiring additional training. Specifically, we formulate and solve a constrained optimization problem that simultaneously selects the most appropriate strategy given the target team's capabilities, and allocates its constituents into appropriate coalitions. We evaluate our approach against several baselines, including some that resemble existing approaches, using detailed numerical simulations, StarCraft II battles, and a multi-robot emergency-response scenario. Our results indicate that our framework consistently outperforms all baselines in terms of requirement satisfaction, resource utilization, and task success rates.

Individual-Level Inverse Reinforcement Learning for Mean Field Games

Downloads

Next from AAMAS 2022

A Mean Field Game Model of Spatial Evolutionary Games

Similar lecture

Learning Noise-Induced Reward Functions for Surpassing Demonstrations in Imitation Learning

Individual-Level Inverse Reinforcement Learning for Mean Field Games

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

A Mean Field Game Model of Spatial Evolutionary Games

Similar lecture

Learning Noise-Induced Reward Functions for Surpassing Demonstrations in Imitation Learning

Downloads