New Zealand

AAMAS 2020

A Very Short Survey and Critique of Multiagent Deep Reinforcement Learning

multiagent learning

survey

reinforcement learning

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

technical paper

Fair Division of the Commons

Encapsulating Reacting Behaviour In Goal-Based Plans For Programming BDI Agents

AAMAS 2020  - Day 1

We study online learning settings in which experts act strategically to maximize their influence on the learning algorithm’s predictions by potentially misreporting their beliefs about a sequence of binary events. Our goal is twofold. First, we want the learning algorithm to be no-regret with respect to the best-fixed expert in hindsight. Second, we want incentive compatibility, a guarantee that each expert’s best strategy is to report his true beliefs about the realization of each event. To achieve this goal, we build on the literature on wagering mechanisms, a type of multi-agent scoring rule. We provide algorithms that achieve no regret and incentive compatibility for myopic experts for both the full and partial information settings. In experiments on datasets from FiveThirtyEight, our algorithms have regret comparable to classic no-regret algorithms, which are not incentive-compatible. Finally, we identify an incentive-compatible algorithm for forward-looking strategic agents that exhibits diminishing regret in practice.

No-Regret and Incentive-Compatible Online Learning

The Gibbard-Satterthwaite theorem states that no non-dictatorial voting rule is strategyproof. We revisit voting rules and consider a weaker notion of strategyproofness called not-obvious manipulability that was proposed by Troyan and Morrill (2019). We identify several classes of voting rules that that satisfy this notion.We also show that several voting rules including k-approval fail to satisfy this property, i.e., satisfy this property. One of our insights is that certain rules are obviously manipulable when the number of alternatives is relatively large as compared to the number of voters. In contrast to the Gibbard-Satterthwaite theorem, many of the rules we examined were not-obviously manipulable. This reflects the relatively easier satisfiability of the notion and the zero information assumption of not-obvious manipulability, as opposed to the perfect information assumption of strategyproofness.

Obvious Manipulability of Voting Rules

Multi-task IRL allows for the possibility that the expert could be switching between multiple ways of solving the same problem, or interleaving demonstrations of multiple tasks. The learner aims to learn the multiple reward functions that guide these ways of solving the problem. We present a new method for multi-task IRL that generalizes the well-known maximum entropy approach to IRL by combining it with the Dirichlet process based clustering of the observed input. This yields a single nonlinear optimization problem, called MaxEnt Multi-task IRL, which can be solved using the Lagrangian relaxation and gradient descent methods. We evaluate MaxEnt Multi-task IRL in simulation on the robotic task of sorting onions on a processing line where the expert utilizes multiple ways of detecting and removing blemished onions. The method is able to learn the underlying reward functions to a good level of accuracy and it improves on the previous best approach to multi-task IRL.

Maximum Entropy Multi-Task Inverse RL

When designing swarm-robotic systems, systematic compar- ison of algorithms from different domains is necessary to determine which is capable of scaling up to handle the target problem size and target oper- ating conditions. Quantitative metrics capable of addressing these needs during the system design process exist for some of the main principles in swarm robotics: scalability, flexibility, and emergence. We define an additional metric to measure robustness along the axes of (1) sensor and actuator noise, (2) fluctuating population sizes, using mathematical curve similarity measures and queueing theory, respectively. We show that robustness analysis is a crucial part of the design process, and that our metric provides intuitive insights into algorithm selection on many real-world problems of interest which are well modeled as object gather- ing problems with (1) temporally varying swarm sizes, (2) and unreliable robots operating in noisy environments.

Robustness Analysis in Large Robot Swarms

Node similarity measures quantify how similar a pair of nodes are in a network. These similarity measures turn out to be an important fundamental tool for many real world applications such as link prediction in networks, recommender systems etc. An important class of similarity measures are local similarity measures. Two nodes are considered similar under local similarity measures if they have large overlap between their neighboring set of nodes. Manipulating node similarity measures via removing edges is an important problem. This type of manipulation, for example, hinders effectiveness of link prediction in terrorists networks. All the popular computational problems formulated around manipulating similarity measures turn out to be NP-hard. We, in this paper, provide fine grained complexity results of these problems through the lens of parameterized complexity. In particular, we show that some of these problems are fixed parameter tractable (FPT) with respect to various natural parameters whereas other problems remain intractable (W[1]-hard and W[2]-hard in particular). Finally we show the effectiveness of our proposed FPT algorithms on real world datasets as well as synthetic networks generated using Barabasi-Albert and ErdosRenyi models.

Manipulating Node Similarity Measures in Networks

What goals should a multi-goal reinforcement learning agent pursue during training in long-horizon tasks? When the desired (test time) goal distribution is too distant to offer a useful learning signal, we argue that the agent should not pursue unobtainable goals. Instead, it should set its own intrinsic goals that maximize the entropy of the historical achieved goal distribution. We propose to optimize this objective by having the agent pursue past achieved goals in sparsely explored areas of the goal space, which focuses exploration on the frontier of the achievable goal set.We show that our strategy achieves an order of magnitude better sample efficiency than the prior state of the art on long-horizon multi-goal tasks including maze navigation and block stacking.

Downloads

Next from AAMAS 2020

Fair Division of the Commons

Similar lecture

Multiagent Climate Change Research

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Fair Division of the Commons

Similar lecture

Multiagent Climate Change Research

Downloads