New Zealand

Kamel Belkhelladi, Pierre Chauvet and Arnaud Schaal (December 1st 2009). Cooperation Control in Distributed Population-based Algorithms Using a Multi-Agent Approach Application to a Real-Life Vehicle Routing Problem, Contemporary Robotics - Challenges and Solutions, A D Rodi&#263;, IntechOpen, DOI: 10.5772/7795.

Miklos Nagy and Maria Vargas-Vera (January 1st 2010). Towards an Automatic Semantic Data Integration: Multi-agent Framework Approach, Semantic Web, Gang Wu, IntechOpen, DOI: 10.5772/7310.

Adel Al-Jumaily and Mohamed Al-Jaafreh (December 1st 2006). Multi-Agent System Concepts Theory and Application Phases, Mobile Robotics, Moving Intelligence, Jonas Buchli, IntechOpen, DOI: 10.5772/4730.

Multi-agent coordination is prevalent in many real-world applications. However, such coordination is challenging due to its combinatorial nature. An important observation in this regard is that agents in the real world often only directly affect a limited set of neighbouring agents. Leveraging such loose couplings among agents is key to making coordination in multi-agent systems feasible.
In this work, we focus on learning to coordinate.
Specifically, we consider the multi-agent multi-armed bandit framework, in which fully cooperative loosely-coupled agents must learn to coordinate their decisions to optimize a common objective. 
We propose multi-agent Thompson sampling (MATS), a new Bayesian exploration-exploitation algorithm that leverages loose couplings. We empirically show that MATS outperforms the state-of-the-art algorithm, MAUCE, on two synthetic benchmarks with Bernoulli-distributed rewards. Our results show that MATS improves significantly upon state-of-the-art coordination methods in terms of performance, demonstrating the value of using MATS in applications with sparse neighbourhood structures.


AAMAS 2020

Thompson Sampling for Factored Multi-Agent Bandits

multi-agent; multi-armed bandits; thompson sampling; coordination graphs

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

technical paper

Despite the success of deep reinforcement learning in a variety type of games such as Board games, RTS, FPS, and MOBA games, sports games (SPG) like basketball have been seldom studied. Basketball is one of the most popular and challenging sports games due to its long-time horizon, sparse rewards, complex game rules, and multiple roles with different capabilities. Although these problems could be partially alleviated by common methods like hierarchical reinforcement learning through a decomposition of the whole game into several subtasks based on game rules (such as attack, defense), these methods tend to ignore the strong correlations between these subtasks and could have difficulty in generating reasonable poli- cies across the whole basketball match. Besides, the existence of multiple agents adds extra challenges to such game. In this work, we propose an integrated curriculum training approach (ICTA) which is composed of two parts. The first part is for handling the correlated subtasks from the perspective of a single player, which contains several weighted cascading curriculum learners that can smoothly unify the base curriculum training of corresponding sub- tasks together using a Q-value backup mechanism with a weight factor. The second part is for enhancing the cooperation ability of the basketball team, which is a curriculum switcher that focuses on learning the switch of the cooperative curriculum within one team by taking over collaborative actions such as passing from single- player’s action spaces. Our method is then applied to a commercial online basketball game named Fever Basketball (FB). Results show that ICTA significantly outperforms the built-in AI and reaches up to around 70% win-rate than online human players during a 300-day evaluation period.


Mastering Basketball with Deep Reinforcement Learning: An Integrated Curriculum Training Approach

We consider Deep Reinforcement Learning (DRL) approaches to devise mapless navigation strategies for mobile platforms. We propose a Genetic Deep Reinforcement Learning (GDRL) method that combines Genetic Algorithms (GA) with discrete and continuous action space DRL approaches. The goal of GDRL is to reduce the sensitivity of DRL approaches to their hyper-parameter tuning and to provide robust exploration strategies. We evaluate GDRL in combination with Rainbow and Proximal Policy Optimization (PPO) in two navigation scenarios: i) a wheeled robot avoiding obstacles in an indoor environment and ii) a water drone that must reach a predefined location in presence of waves. Our empirical evaluation demonstrates that GDRL outperforms state-of-the-art DRL and GA methods as well as a previous hybrid approach.


Genetic Deep Reinforcement Learning for Mapless Navigation

Multi-agent path finding (MAPF), naturally exists in applications like picking-up and dropping-off parcels by automated guided vehicles (AGVs) in the warehouse. Existing algorithms, like conflict-based search (CBS), windowed hierarchical cooperative A* (WHCA), and other A* variants, are widely used to find the shortest paths in different manners. However, in real-world environments, MAPF cases are dynamically generated and need to be solved in real time. In this work, a decentralized multi-agent reinforcement learning (MARL) framework with multi-step ahead tree search (MATS) strategy is proposed to make efficient decisions. Through performing experiments on a 30*30 grid world and a real-world warehouse case, our proposed MARL policy is proved to be capable of: 1) scaling to a large number of agents in real-world environment with online response time within acceptable levels; 2) outperforming existing algorithms with shorter path length and solution time, as the number of agents increases.


Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding

In this talk, we study a variant of hedonic games, called Seat Arrangement. 
The model is defined by a bijection from agents with preferences to vertices in a graph.
The utility of an agent depends on the neighbors in the graph. 
In this talk, we study the price of stability and fairness in Seat Arrangement, and the computational complexity and the parameterized complexity of finding certain desirable seat arrangements, say Maximum Welfare Arrangement, Maximin Utility Arrangement, Stable Arrangement, and Envy-free Arrangement.


Hedonic Seat Arrangement Problems

In order to be useful in the real world, an AI agent needs to plan and act in the presence of other agents, who may be helpful or disruptive. In this paper, we consider the problem where an autonomous agent needs to act in a manner that clarifies its objectives to cooperative agents while simultaneously preventing adversarial agents from inferring those objectives. We call it Mixed-Observer Controlled Observability Planning Problem (MO-COPP). We develop two new solution approaches: one provides an optimal solution to the problem given a fixed time horizon by using an integer programming solver, the other provides a satisficing solution using heuristic-guided forward search to achieve prespecified amount of obfuscation and legibility for adversarial and cooperative agents respectively. 


Signaling Friends and Head-Faking Enemies Simultaneously: Balancing Goal Obfuscation and Goal Legibility

In many multi-agent interactions in the real world, agents receive payoffs over multiple distinct criteria; i.e. the payoffs are multi-objective in nature. However, the same multi-objective payoff vector may lead to different utilities for each participant. Therefore, it is essential for an agent to learn about the behaviour of other agents in the system. We investigate here the effects of opponent modelling on multi-objective multi-agent interactions with non-linear utilities. Specifically, we consider multi-objective normal form games (MONFGs) with non-linear utility functions under the scalarised expected returns optimisation criterion. We contribute a novel actor-critic formulation to allow reinforcement learning of mixed strategies in this setting, along with an extension that incorporates opponent policy reconstruction using conditional action frequencies. Our empirical results demonstrate that opponent modelling can drastically alter the learning dynamics in this setting.


Opponent Modelling for Reinforcement Learning in Multi-Objective Normal Form Games

Too many coocks: Coordinating multi-agent collaboration through inverse planning

Reinforcement learning and planning have been revolutionized in recent years, due in part to the mass adoption of deep convolutional neural networks and the resurgence of powerful methods to refine decision-making policies. However, the problem of sparse reward signals remains pervasive in many domains. While various reward-shaping mechanisms and imitation learning approaches have been proposed to mitigate this problem, a mathematically rigorous structure of the underlying objective is rarely exploited. In this paper, we resolve this by defining objectives using linear temporal logic over finite traces (LTL_f) and utilize the automaton representation of said objectives in order to define novel reward shaping functions that mitigate the sparse rewards problem within modern Monte Carlo Tree Search (MCTS) methods. We further demonstrate that such verification-guided reward shaping can be utilized to facilitate transfer learning between different environments when the objective is the same.


Verification-Guided Tree Search

This presentation describes the use of CArtAgO artifacts to represent virtual resources exposed by an agent in a hypermedia context as an extension of the Multi-Agent MicroServices architecture.

Explicit Modelling of Resources for Multi-Agent Microservices using the CArtAgo framework

Simulated games have become a dominant platform for multiagent intelligence research in recent years. Previous works have succeeded on arcade, first person shooter (FPS), real-time strategy (RTS), and massive online battle arena (MOBA) games. Our work considers massively multiplayer online role-playing games (MMORPGs or MMOs), which capture several complexities of real-world learning that are not well modeled by any other game genre. We present a massively multiagent game environment inspired by MMOs and demonstrate that simple policy gradient methods produce interesting emergent exploration and specialization behaviors.


Downloads

Next from AAMAS 2020

Mastering Basketball with Deep Reinforcement Learning: An Integrated Curriculum Training Approach

Similar lecture

Thompson Sampling for Factored Multi-Agent Bandits

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Mastering Basketball with Deep Reinforcement Learning: An Integrated Curriculum Training Approach

Similar lecture

Thompson Sampling for Factored Multi-Agent Bandits

Downloads