New Zealand


In MARL, communication among agents is essential to establish cooperation. Over the realistic wireless network, many factors can affect transmission reliability, especially considering that the wireless network condition varies with agents’ mobility. We propose a framework that improves the intelligence of communication over realistic wireless networks in two fundamental aspects: (1) When: Agents learn the timing of communication based on message importance and wireless channel condition. We further propose a communication lagging technique to make the training end-to-end differentiable. (2) What: Agents augment message contents with
wireless network measurements. The messages improve both the game and communication actions of the agents. Experiments on a standard environment show that compared with state-of-the-art, our framework enables more intelligent collaboration and thus achieves significantly better game performance, convergence speed and communication efficiency.

AAMAS 2022

Intelligent Communication over Realistic Wireless Networks in Multi-Agent Cooperative Games

wireless communication

multi-agent reinforcement learning

collaboration

The IFAC Conference on Nonlinear Model Predictive Control (NMPC 2021) aims to bring together researchers interested in and work in the MPC field from both academia and industry. This allows us to reflect and establish the current state-of-the-art and focus on the future development of the MPC field towards relevant directions.

Model predictive control (MPC) is indisputably one of the advanced control techniques that have significantly affected control engineering practice with thousands of controllers implemented in various fields, spanning from the process industry to automotive and robotics. This is due to its unique ability to handle constraints, take preview information about disturbances and setpoint changes into account, and optimize performance. Furthermore, MPC allows the consideration of various types of dynamics, from linear to nonlinear, discrete to continuous, as well as an online adaptation of models and parameters. The research community on MPC is very active, with hundreds of papers published every year. Over the past decades, many theoretical, as well as implementation results, have been achieved.

### **All content presented at NMPC2021, including videos, papers and slides will be available only to registered (regular/student) participants. The content can be accessed until August 31st, 2021.**



![](https://assets.underline.io/uploads/markdown_image/1/image/946cb7978767129748bc1bb6270cb362.png)

Please register with NMPC 2021 to be able to enter the conference on Underline.io. 
 If you have already registered, please login to Underline.io to be able to see full conference content. 
 In case you do not have an Underline account, please create one. Thank you

Please login to your Underline account

Nonlinear Model Predictive Control Conference 2021

The IFAC Conference on Nonlinear Model Predictive Control (NMPC 2021) aims to bring together researchers interested in and work in the MPC field from both academia and industry. This allows us to reflect and establish the current state-of-the-art and focus on the future development of the MPC field towards relevant directions.

poster

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).

A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

Federated learning (FL) allows distributed clients to collaboratively train a global model without sharing their private data. We propose a novel FL algorithm, which can accurately evaluate the contributions of clients and aggregate global model in a more efficient manner. More specifically, we propose to use a Deep Reinforcement Learning (DRL) method to dynamically learn the contributions of clients in each communication round. Based on this, we adaptively assign appropriate weights to clients, which will be used in global model aggregation. By improving the process of global mode aggregation, our proposed scheme greatly improves the performance of federated learning.

Adaptive Aggregation Weight Assignment for Federated Learning: A Deep Reinforcement Learning Approach


PointGoal navigation has seen significant recent interest and progress,
spurred on by the Habitat platform and associated challenge.
In this paper, we study PointGoal navigation under both a sample
budget (75 million frames) and a compute budget (1 GPU for 1 day).
We conduct an extensive set of experiments, cumulatively totaling
over 50,000 GPU-hours, that let us identify and discuss a number of
ostensibly minor but significant design choices – the advantage esti-
mation procedure (a key component in training), and visual encoder
architecture. Overall, these design choices lead to considerable and
consistent improvements. Under a sample budget, performance for
RGB-D agents improves 3 SPL on Gibson (4% relative improvement)
and 20 SPL on Matterport3D (43% relative improvement). Under a
compute budget, performance for RGB-D agents improves by 3 SPL
on Gibson (5% relative improvement) and 15 SPL on Matterport3D
(50% relative improvement). Our findings and recommendations
will serve to make the community’s experiments more efficient – to
reach 50 SPL with RGB-D on Matterport3D, they reduce the samples
needed by 3x and the training time 2x.

How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget

Multiagent self-assembly allows collectives to reach areas otherwise inaccessible to any particular agent. However, the coordination of this collective is not trivial, so each agent’s position in the structure is usually determined apriori. In our approach, we take inspiration from army ants and use a simulated model of the Eciton Robotica robot to form emergent structures with bio-inspired local rules. We demonstrate that by coupling this with traffic control, we can induce the formation of a structure and control certain characteristics without pre-computed paths or central coordination.

Influencing Emergent Self-Assembled Structures in Robotic Collectives Through Traffic Control

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1536.pdf

**Abstract:**



Can Algorithms be Explained Without Compromising Efficiency? The Benefits of Detection and Imitation in Strategic Classification


Representation learning on visualized input is an essential yet challenging task for deep reinforcement learning (RL). While training agents using deep reinforcement learning algorithms, we usually assume that the agent could extract appropriate features and take actions accordingly. However, learning essential and generalizable features based on high-dimensional input images is a great challenge. To help the RL agent learn more general and discriminative representation
among various states, we present cross-state self-constraint (CSSC). This novel technique regularizes representation learning by comparing state embedding similarities across different state-action pairs. We test our proposed method on the OpenAI Procgen benchmark with Rainbow and PPO and demonstrate significant improvement across most Procgen environments.

Improving Generalization with Cross-State Behavior Matching in Deep Reinforcement Learning


Measuring and comparing resilience is crucial for evaluating different algorithms’ performance. Existing resilience metrics focus on a system’s ability to maintain a particular state, but are inadequate to evaluate whether a system can achieve a novel state after an unexpected disturbance. The presented resilience power metric is used to analyze two best-of-N algorithms. Both algorithms exhibited high resilience power when changing the collective’s population size, but this result did not correlate with high overall task performance.

Measuring Resilience in Collective Robotic Algorithms


Covering option discovery has been developed to improve the exploration of reinforcement learning in single-agent scenarios with sparse reward signals, through connecting the most distant states in the embedding space provided by the Fiedler vector of the state transition graph. However, due to exponentially-large state space
resulted from multi-agent scenarios, existing researches on adopting options in multi-agent scenarios still rely on single-agent option discovery and fail to directly discover the joint options that can improve the connectivity of the joint state space of agents. In this paper, we show that it is indeed possible to directly compute
multi-agent options with collaborative exploratory behavior while still enjoying the ease of decomposition. Our key idea is to approximate the joint state space as a kronecker graph – the kronecker product of individual agents’ state transition graphs, based on which we can directly estimate the Fiedler vector of the joint
state space using the Laplacian spectrum of individual agents’ transition graphs. This decomposition enables us to efficiently construct multi-agent joint options by encouraging agents to connect the sub-goal joint states which are corresponding to the minimum or maximum values of the estimated joint Fiedler vector. The evaluation based on multi-agent collaborative tasks shows that the proposed
algorithm can successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options, in terms of both faster exploration and higher cumulative rewards.

Multi-agent Covering Option Discovery through Kronecker Product of Factor Graphs

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1592.pdf

**Abstract:**



Beyond Uninformed Search: Improving Branch-and-bound Based Acceleration Algorithms for Belief Propagation via Heuristic Strategies

We study mechanisms that allocate reviewers to papers in a fair and efficient manner. We model reviewer assignment as an instance of a fair allocation problem, presenting an extension of the classic round-robin mechanism, called Reviewer Round Robin (RRR). Round-robin mechanisms are a standard tool to ensure envy-free up to one item (EF1) allocations. However, fairness often comes at the cost of decreased efficiency. To overcome this challenge, we carefully select an approximately optimal round-robin order. Applying a relaxation of submodularity, $\gamma$-weak submodularity, we show that greedily inserting papers into an order yields a ${(1+\gamma^2)}$-approximation to the maximum welfare attainable by our round-robin mechanism under any order. Our approach outputs highly efficient EF1 allocations for three real conference datasets, offering comparable performance to state-of-the-art paper assignment methods in fairness, efficiency, and runtime, while providing the only EF1 guarantee.

I Will Have Order! Optimizing Orders for Fair Reviewer Assignment


Evacuation plans are designed to move people to safety in case of a disaster. It mainly consists of two components: routing and scheduling. However, the joint optimization of these two components with the goal of minimizing total evacuation time is a computationally hard problem, specifically when the problem instance is large. Moreover, often in disaster situations, there is uncertainty regarding the passability of roads throughout the evacuation time period. In this paper, we present a way to model the time-varying risk associated with roads in disaster situations. Then, we present a heuristic method that performs the joint optimization over routing and scheduling by iteratively solving subproblems of the same form. We use real world road network and population data from Harris County in Houston, Texas and apply our heuristic to find evacuation routes and schedules for the area. We show that the proposed method is able to find good solutions within a reasonable amount of time. We also perform agent-based simulations of the evacuation using these solutions to evaluate their quality and efficacy.

Downloads

Next from AAMAS 2022

Adaptive Aggregation Weight Assignment for Federated Learning: A Deep Reinforcement Learning Approach

Similar lecture

Robust MPC for Networks with Varying Communication Capabilities

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

Adaptive Aggregation Weight Assignment for Federated Learning: A Deep Reinforcement Learning Approach

Similar lecture

Robust MPC for Networks with Varying Communication Capabilities

Downloads