New Zealand


Emergent communication has made strides towards learning communication from scratch, but has thus far only focused on protocols that resemble human language. In nature, multi-agent cooperation gives rise to a wide range of communication that varies in structure and complexity. In this work, we recognize the full spectrum of communication that exists in nature and propose studying lower-level communication. Specifically, we study emergent implicit communication in the context of decentralized multi-agent learning in difficult, sparse reward environments. However, learning to coordinate in such environments is challenging. To this end, we propose a curriculum-driven multi-agent strategy that combines: (i) velocity-based environment shaping, tailored to the skill level of the multi-agent team; and (ii) a behavioral curriculum that helps agents learn successful single-agent behaviors as a precursor to learning multi-agent behaviors. Pursuit-evasion experiments show that our approach learns effective coordination, significantly outperforming sophisticated analytical and learned policies. The policies learned by our method complete the pursuit-evasion task when pursuers move at only one-half of the evader&#39;s speed, whereas the highest-performing baseline fails at $80\%$ of the evader&#39;s speed. Moreover, we examine the use of implicit signals in coordination through position-based social influence. We show that pursuers trained with our strategy exchange more than twice as much information (in bits) than baseline methods, indicating that our method has learned, and relies heavily on, the exchange of implicit signals.

AAMAS 2022

Multi-Agent Curricula and Emergent Implicit Signaling

multi-agent systems

communication

reinforcement learning

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).
<br>
<br>



A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).


Emergent communication has made strides towards learning communication from scratch, but has thus far only focused on protocols that resemble human language. In nature, multi-agent cooperation gives rise to a wide range of communication that varies in structure and complexity. In this work, we recognize the full spectrum of communication that exists in nature and propose studying lower-level communication. Specifically, we study emergent implicit communication in the context of decentralized multi-agent learning in difficult, sparse reward environments. However, learning to coordinate in such environments is challenging. To this end, we propose a curriculum-driven multi-agent strategy that combines: (i) velocity-based environment shaping, tailored to the skill level of the multi-agent team; and (ii) a behavioral curriculum that helps agents learn successful single-agent behaviors as a precursor to learning multi-agent behaviors. Pursuit-evasion experiments show that our approach learns effective coordination, significantly outperforming sophisticated analytical and learned policies. The policies learned by our method complete the pursuit-evasion task when pursuers move at only one-half of the evader's speed, whereas the highest-performing baseline fails at $80\%$ of the evader's speed. Moreover, we examine the use of implicit signals in coordination through position-based social influence. We show that pursuers trained with our strategy exchange more than twice as much information (in bits) than baseline methods, indicating that our method has learned, and relies heavily on, the exchange of implicit signals.

technical paper


We address the scaling of equilibrium computation in Mean Field Games (MFGs) by using Online Mirror Descent (OMD). We show that continuous-time OMD provably converges to a Nash equilibrium under a natural and well-motivated set of monotonicity assumptions. A thorough experimental investigation on various single, and multi-population, MFGs shows that OMD outperforms traditional algorithms such as Fictitious Play (FP). We empirically show that OMD scales and converges significantly faster than FP by solving, for the first time to our knowledge, examples of MFGs with hundreds of billions states. As such, this study establishes the state-of-the-art for learning in large-scale multi-agent and multi-population games.

Scaling Mean Field Games with Online Mirror Descent


During the COVID-19 pandemic, governments have struggled to devise strategies to slow down the spread of the virus. This struggle happens because pandemics are complex scenarios with many unknown variables. In this context, simulated models are used to evaluate strategies for mitigating this and future pandemics. In this paper, we propose a simulator that analyses small communities by using real geographical data to model the road interactions and the agent's behaviors. Our simulator consists of three different modules: Environment, Mobility, and Infection module. The environment module recreates an area based on map data, including houses, restaurants, and roads. The mobility module drives the agents to move in the map based on their work schedule and needs, such as eating at restaurants, doing groceries, and going to work. The infection module simulates four cases of infection: on the road, at home, at a building, and off the map. We simulate the surrounding areas of a university campus. We design four intervention strategies and compare them to a scenario without any intervention. The interventions are: 1) PCR testing and self-isolation if positive; 2) reducing working hours at barbershops and restaurants; 3) closing barbershops and restaurants; 4) closing grocery stores and restaurants and providing delivery instead. For all scenarios, we observe the two types of areas where most infection happens: hubs, where people from different occupations can meet (e.g., restaurants); and non-hubs, where people with the same occupation meet (e.g., offices). The simulations show that all interventions reduce the total number of infected agents by a large margin. We observed that interventions targeting hubs (2-4) did not impact the infection at non-hubs. In addition, the intervention targeting people's behavior (1) ended up creating a cluster at the testing center.

Using Agent-Based Simulator to Assess Interventions Against COVID-19 in a Small Community Generated from Map Data


In response to COVID-19, many countries have mandated social distancing and banned large group gatherings in order to slow down the spread of SARS-CoV-2. Despite the effectiveness of these mandates, the only way to eradicate the disease is to achieve herd immunity through vaccination. In order to increase the accessibility of vaccines, states such as Virginia have funded mobile vaccination centers to distribute vaccines across the state. When choosing where to place these sites, there are two important factors to take into account: accessibility and equity. We develop efficient algorithms with theoretical guarantees on both of these aspects. Finally, we run computational experiments on real world data to show the efficacy of our methods.

Deploying Vaccine Distribution Sites for Improved Accessibility and Equity to Support Pandemic Response


Classification and clustering are prevalent to recognize the identity and the community of nodes in graphs. Several studies analyzed graph structure to build graph neural network (GNN) models for reducing the accuracy of node classification and clustering recently. The existing defense usually modified the structure of models and adopted countermeasure training to enhance the robustness of the node classification and clustering. However, these defense works are model-oriented and not robust. To alleviate the problem, this paper first proposed a robust node classification metric based on residual entropy. More concrete, the accuracy of node classification is proven robustly by maximizing the residual entropy of the nodes theoretically. Then two graph generative algorithms are proposed to resist against two kinds of GNN-based attacks, the untargeted attack and the targeted attack, respectively. Finally, experimental analysis show that the proposed algorithms outperform the existing defense works under five classic datasets.

Residual Entropy-based Graph Generative Algorithms


Agent-based models (ABMs) highlight the importance of simulation validation, such as qualitative face validation and quantitative empirical validation. In particular, we focused on quantitative validation by adjusting simulation input parameters of the ABM. This study introduces an automatic calibration framework that combines the suggested dynamic and heterogeneous calibration methods. Specifically, the dynamic calibration fits the simulation results to the real-world data by automatically capturing suitable simulation time to adjust the simulation parameters, which continuously regulates the simulation diverging from the real world. Conversely, the heterogeneous calibration fits the distributional discrepancy between individuals in simulation and the real world by adjusting agent related parameters cluster-wisely.

Automatic calibration framework of agent-based models for dynamic and heterogeneous parameters


We present a logic of responsibility. Extending stit theory with epistemic, doxastic, deontic, and intentional modalities, we provide logic-based characterizations of several modes of responsibility.

A Stit Logic of Responsibility

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1607.pdf

**Abstract:**



A Graph Neural Network Reasoner for Game Description Language

Existing approaches to coalition formation often assume that requirements associated with tasks are precisely specified by the human operator. However, prior work has demonstrated that humans, while extremely adept at solving complex problems, struggle to explicitly state their solution strategy. Further, existing approaches often ignore the fact that experts may utilize different, but equally-valid, solutions (i.e., heterogeneous strategies} to the same problem. In this work, we propose a two-part framework to address these challenges. First, we tackle the challenge of inferring implicit strategies directly from expert demonstrations of coalition formation. To this end, we model and infer such heterogeneous strategies as capability-based requirements associated with each task. Next, we propose a method capable of adaptively selecting one of the inferred strategies that best suits the target team without requiring additional training. Specifically, we formulate and solve a constrained optimization problem that simultaneously selects the most appropriate strategy given the target team's capabilities, and allocates its constituents into appropriate coalitions. We evaluate our approach against several baselines, including some that resemble existing approaches, using detailed numerical simulations, StarCraft II battles, and a multi-robot emergency-response scenario. Our results indicate that our framework consistently outperforms all baselines in terms of requirement satisfaction, resource utilization, and task success rates.

Resource-Aware Adaptation of Heterogeneous Strategies for Coalition Formation

Diversity in behaviors is instrumental for robust team performance in many multiagent tasks which require agents to coordinate. Unfortunately, exhaustive search through the agents' behavior spaces is often intractable.
This paper introduces Behavior Exploration for Heterogeneous Teams (BEHT), a multi-level learning framework that enables agents to progressively explore regions of the behavior space that promote team coordination on diverse goals.
By combining diversity search to maximize agent-specific rewards and evolutionary optimization to maximize the team-based fitness, our method effectively filters regions of the behavior space that are conducive to agent coordination.
We demonstrate the diverse behaviors and synergies that are method allows agents to learn on a multiagent exploration problem.

Behavior Exploration and Team Balancing for Heterogeneous Multiagent Coordination

Efficient contact tracing and isolation is an effective strategy to control epidemics, as seen in the Ebola epidemic and COVID-19 pandemic. An important consideration in contact tracing is the budget on the number of individuals asked to quarantine---the budget is limited for socioeconomic reasons (e.g., having a limited number of contact tracers). Here, we present a Markov Decision Process (MDP) framework to formulate the problem of using contact tracing to reduce the size of an outbreak while limiting the number of people quarantined. We formulate each step of the MDP as a combinatorial problem, MinExposed, which we demonstrate is NP-Hard. Next, we develop two approximation algorithms, one based on rounding the solutions of a linear program and another (greedy algorithm) based on choosing nodes with a high (weighted) degree. A key feature of the greedy algorithm is that it does not need complete information of the underlying social contact network, making it implementable in practice. Using simulations over realistic networks, we show how the algorithms can help in {\em bending} the epidemic curve with a limited number of isolated
individuals.

Multi-Agent Curricula and Emergent Implicit Signaling

Downloads

Next from AAMAS 2022

Scaling Mean Field Games with Online Mirror Descent

Similar lecture

GCS: Graph-Based Coordination Strategy for Multi-Agent Reinforcement Learning

Multi-Agent Curricula and Emergent Implicit Signaling

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

Scaling Mean Field Games with Online Mirror Descent

Similar lecture

GCS: Graph-Based Coordination Strategy for Multi-Agent Reinforcement Learning

Downloads