New Zealand

Mengchun Xie (March 18th 2019). Improvement of Cooperative Action for Multi-Agent System by Rewards Distribution, Assistive and Rehabilitation Engineering, Yves Rybarczyk, IntechOpen, DOI: 10.5772/intechopen.85109.

Dennis Barrios-Aranibar and Luiz M. G. Gon&#231;alves (January 1st 2009). Influence Value Q-Learning: A Reinforcement Learning Algorithm for Multi Agent Systems, Theory and Novel Applications of Machine Learning, Meng Joo Er and Yi Zhou, IntechOpen, DOI: 10.5772/6675. 

Bart&#322;omiej &#346;nie&#380;y&#324;ski (January 1st 2009). Supervised Rule Learning and Reinforcement Learning in A Multi-Agent System for the Fish Banks Game, Theory and Novel Applications of Machine Learning, Meng Joo Er and Yi Zhou, IntechOpen, DOI: 10.5772/6672. 

We develop a Multi-Agent Reinforcement Learning (MARL) method that finds approximately optimal policies for cooperative agents that co-exist in an environment. Central to achieving this is how the agents learn to communicate with each other. Can they together develop a language while learning to perform a common task? We formulate and study a MARL problem where cooperative agents are connected via a fixed underlying network. These agents communicate along the edges of this network by exchanging discrete symbols. However, the semantics of these symbols are not predefined and have to be learned during the training process. We propose a method for training these agents using emergent communication. We demonstrate the applicability of the proposed framework by applying it to the problem of managing traffic controllers, where we achieve state-of-the-art performance (as compared to several strong baselines) and perform a detailed analysis of the emergent communication.


AAMAS 2020

Networked Multi-Agent Reinforcement Learning with Emergent Communication

traffic management

multi-agent reinforcement learning

emergent communication

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

technical paper

Humans use language to collectively execute abstract strategies besides using it as a referential tool for identifying physical entities. In our paper, we study the role of emergent languages in discovering and implementing strategies. We formulate the problem using a voting game where two candidate agents contest in an election with the goal of convincing population members (other agents), that are connected to each other via an underlying network, to vote for them. To achieve this goal, agents are only allowed to exchange messages in the form of sequences of discrete symbols. Using our proposed framework, we answer the following questions: (i) Do the agents learn to communicate in a meaningful way? (ii) Does the system evolve as expected under various reward structures? (iii) How is the emergent language affected by the community structure in the network? To the best of our knowledge, we are the first to study emergence of communication among networked agents for discovering and implementing strategies.

Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks

In many practical domains, applying machine learning is challenging not due to the lack of available data, but because labeled samples are in short supply. A common approach for obtaining additional labeled samples is co-training, a semi-supervised learning setting where two learners (agents), trained on different perspectives of the data, iteratively label additional samples. The rationale of this approach is that the different learner perspectives will produce a more diverse labeled set, resulting in more effective classifiers. While co-training proved effective in multiple cases, the labeling mechanisms used by existing approaches are heuristic and error-prone. We propose CoMet, a meta learning-based co-training algorithm. CoMet utilizes meta-models trained on previously analyzed datasets to select the samples to be labeled for the current dataset. Our experiments, conducted on 35 datasets, show that CoMet significantly outperforms the standard co-training approach.


CoMet: A Meta Learning-Based Approach for Cross-Dataset Labeling Using Co-Training

Testing modern computer games is notoriously hard. Highly dynamic behavior, inherent non-determinism, and fine grained interactivity blow up their state space; too large for traditional automated testing techniques. An agent-based testing approach offers an alternative as agents' goal driven planning, adaptivity, and reasoning ability can provide an extra edge. 
We present aplib, a Java library for programming intelligent test agents, featuring tactical programming as an abstract way to exert control on agents' underlying reasoning based behavior. Aplib is implemented in such a way to provide the fluency of a Domain Specific Language (DSL) while still staying in Java, and hence aplib programmers will keep all the advantages that Java programmers get: rich language features and a whole array of development tools.

APLIB: An Agent Programming Library for Testing Games

When dealing with election data it is reasonable to assume that the votes are incomplete or noisy. The reasons are manifold and range from cost-intensive elicitation to manipulation. We study the important questions of evaluating elections with incomplete data and the robustness of elections with noisy data from a computational point of view. To capture different motivations, we consider three models for the distribution of preferences: the uniform distribution over the completions of incomplete preferences inspired by the possible winner problem, the dispersion around complete preferences, also called Mallows noise model, and a model in which the distribution over the votes of each voter is explicitly given. We consider both approval vector preferences and linear order preferences and show that the complexity of the problem can vary greatly depending on the voting rule, the distribution model, and the parameterization.


Complexity of Election Evaluation and Probabilistic Robustness

In this paper, we propose a self-adaptive approach to build a smart traffic light management dealing with intersections. This approach relies on the multiagent systems architecture, suitable to support a distributed and collaborative mechanism of regulation while taking into account dynamic changes in the traffic flow. In our solution, the agents model the intersections and can decide how long is the duration of traffic lights according to their perception of the traffic flow. Each intersection agent uses a behavior tree to update the traffic light status (i.e. switch from green to red lights and vice-versa), changing the duration of each status dynamically, according to the number of cars perceived in each intersection. We also demonstrate how dynamic traffic control policies can be used in a collaborative scenario to regulate traffic flow. 


Adaptive and Collaborative Agent-based Traffic Regulation Using Behavior Trees

We present an overview of the Task Coordination (TC) problem in multiagent systems and discuss the specific elements that are required to develop a solution to this problem. Task coordination refers to a twofold problem where an exogenously imposed state of affairs should be satisfied by a multiagent system (MAS): (1) the agents need to be assigned tasks to fulfill the given state of affairs (task allocation) and (2) the behavior of agents needs to be monitored to evaluate whether their tasks are fulfilled so that responsibility for dismissed tasks can be determined (task responsibility). This becomes especially challenging when agents are autonomous and may have imperfect information about their environment. Then, the allocation of tasks and responsibilities should regard agents' strategic ability under imperfect information. To date, existing work on the application of strategic reasoning for task allocation assumes perfect information for agents (dismissing imperfect information settings) and allocates tasks to individual agents (dismissing the potential for allocating tasks to agent groups). This calls for TC frameworks able to model task allocation in imperfect information settings and by allowing the allocation of tasks to agent groups. Such a framework should also be able to determine the responsibility of agents for dismissed tasks via a task responsibility mechanism that complements the task allocation procedure. This work discusses various aspects of the TC problem, sets forward a conceptual analysis on expected properties of potential solution concepts, and presents the overview of a suggested approach for developing a TC framework using techniques from formal strategic reasoning.


Task Coordination in Multiagent Systems

Although plenty of qualitative logical frameworks have been pro- posed to evaluate and model trust in multi-agent sittings, these ap- proaches generally ignore reasoning about quantitative aspects such as degrees of trust. In this paper, we address this limitation from the modelling and verification perspectives. We start by constructing TCTLG , a logical language to represent the quantitative aspect of trust and present a set of its reasoning postulates. Moreover, we develop and implement a new symbolic model checking algorithm and open source tool for quantifying the relationships among the interacting agents. Finally, we investigate the complexity and evaluate our approach using a case study in the health care domain.


Computationally Grounded Quantitative Trust with Time

Defenders in security problems often use anomaly detection (AD) to examine effects of (adversarial) actions and detect malicious behavior. Attackers seek to accomplish their goal (e.g., exfiltrate data) while avoiding the detection. Game theory can be used to reason about this interaction. While AD has been used in game-theoretic frameworks before, we extend the existing works to more realistic settings by (1) allowing players to have continuous action spaces and (2) assuming that the defender cannot perfectly observe the action of the attacker. We solve our model by (1) extending existing algorithms that discretize the action spaces and use linear programming and (2) by training a neural network using an algorithm based on exploitability descent, termed EDA. While both algorithms are applicable for low feature-space dimensions, EDA produces less exploitable strategies and scales to higher dimensions. In a data exfiltration scenario, EDA outperforms a range of classifiers when facing a targeted exploitative attacker.


Discovering Imperfectly Observable Adversarial Actions using Anomaly Detection

In the multidimensional stable roommate problem, agents have to be allocated to rooms and have preferences over sets of potential roommates. We study the complexity of finding good allocations of agents to rooms under the assumption that agents have diversity preferences [13]: each agent belongs to one of the two types (e.g., juniors and seniors, artists and engineers), and agents’ preferences over rooms depend solely on the fraction of agents of their own type among their potential roommates. We consider various solution concepts for this setting, such as core and exchange stability, Pareto optimality and envy-freeness. On the negative side, we prove that envy-free, core stable or (strongly) exchange stable outcomes may fail to exist and that the associated decision problems are NP-complete. On the positive side, we show that these problems are in FPT with respect to the room size, which is not the case for the general stable roommate problem.


Stable Roommate Problem with Diversity Preferences

In this paper, we relate influence maximisation (IM) for the voting dynamics to models of network control in which external controllers interact with the intrinsic dynamics of opinion spread. In contrast to previous literature, which has mostly explored the discrete
setting, our focus is on continuous allocations of control. We develop an algorithm to numerically solve our IM problem via gradient ascent. We explore optimal allocations for leader-follower type networks for different budget scenarios and observe that optimal
allocations do not systematically target hub nodes, as it has been found in previous literature. Conversely, strategies are strongly opponent-depend, avoiding nodes targeted by the opponent if the opponent has a larger budget, while shadowing the opponent’s
allocation otherwise, i.e. targeting the same nodes as them.


Downloads

Next from AAMAS 2020

Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks

Similar lecture

Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks

Similar lecture

Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks

Downloads