New Zealand

Toshihiko Watanabe (February 1st 2010). Hierarchical Reinforcement Learning Using a Modular Fuzzy Model for Multi-Agent Problem, New Advances in Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/9388. 

Masanao Obayashi, Kenichiro Narita, Yohei Okamoto, Takashi Kuremoto, Kunikazu Kobayashi and Liangbing Feng (January 14th 2011). A Reinforcement Learning System Embedded Agent with Neural Network-Based Adaptive Hierarchical Memory Structure, Advances in Reinforcement Learning, Abdelhamid Mellouk, IntechOpen, DOI: 10.5772/13199.

Le Tien Dung, Takashi Komeda and Motoki Takagi (January 1st 2009). Solving POMDPs with Automatic Discovery of Subgoals, Theory and Novel Applications of Machine Learning, Meng Joo Er and Yi Zhou, IntechOpen, DOI: 10.5772/6685. 

Hierarchical Reinforcement Learning (HRL) is a promising approach to solve more complex tasks which may be challenging for the traditional reinforcement learning. HRL achieves this by decomposing a task into shorter-horizon subgoals as targets to achieve. Autonomous discovery of such subgoals is an important part of HRL. Recently, end-to-end HRL methods have been used to reduce the overhead from offline subgoal discovery by seeking the useful subgoals while simultaneously learning optimal policies in a hierarchy. However, these methods may still suffer from slow learning when the search space used by a high level policy to find the subgoals is large. We propose LIDOSS, an end-to-end HRL method with an integrated heuristic for subgoal discovery, to address this problem.

AAMAS 2020

Hierarchical Reinforcement Learning with Integrated Discovery of Salient Subgoals

hierarchical reinforcement learning; reinforcement learning; subgoal discovery

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Silly rules improve the capacity of agents to learn stable enforcement and compliance behaviors

This paper studies the complexity of two microbribery problems under the model of group identification. In these problems, we are given a subset of distinguished individuals, and the questions are whether these individuals can be made socially qualified or whether they can be made exactly the socially qualified individuals, respectively, by modifying a limited number of entries in the qualifications-profile. For consent rules, the consensus-start-respecting rule, and the liberal-start-respecting rule, we obtain many $\np$-hardness results as well as polynomial-time solvability results. We also study the problems in $r$-profiles where each individual qualifies exactly $r$ individuals.

Microbribery in Group Identification

Multi-Agent Path Finding (MAPF) is the problem of moving a team of agents from their start locations to their goal locations without collisions. We study the lifelong variant of MAPF where agents are constantly engaged with new goal locations, such as in warehouses. We propose a new framework for solving lifelong MAPF by decomposing the problem into a sequence of Windowed MAPF instances, where a Windowed MAPF solver resolves collisions among the paths of agents only within a finite time horizon and ignores collisions beyond it. Our framework is particularly well suited to generating pliable plans that adapt to continually arriving new goal locations. We evaluate our framework with a variety of MAPF solvers and show that it can produce high-quality solutions for up to 1,000 agents, significantly outperforming existing methods.

Lifelong Multi-Agent Path Finding in Large-Scale Warehouses

"Hierarchical clustering is a stronger extension of one of today's most influential unsupervised learning methods: clustering. The goal of this method is to create a hierarchy of clusters, thus constructing cluster evolutionary history and simultaneously finding clusterings at all resolutions. We propose four traits of interest for hierarchical clustering algorithms: (1) empirical performance, (2) theoretical guarantees, (3) balance (the minimum ratio between cluster sizes), and (4) scalability. While a number of algorithms are designed to achieve one to two of these traits at a time, there exist none that achieve all four.



Inspired by Bateni et al.'s scalable and empirically successful Affinity Clustering [NeurIPs 2017], we introduce Affinity's successor, Matching Affinity Clustering. Like its predecessor, Matching Affinity Clustering maintains strong empirical performance, even outperforming Affinity when the dataset is size 2^n and clusters are balanced, and uses Massively Parallel Communication as its distributed model. Designed to maintain provably balanced clusters, we show that our algorithm achieves a (1/3-epsilon)-approximation for Moseley and Wang's revenue (the dual to Dasgupta's cost) when the data set is of size 2^n, and a (1/9-epsilon)-approximation in general. We prove the former approximation is tight, and also that Affinity Clustering cannot do better than a 1/O(n)-approximation. In addition, we see that our algorithm empirically performs similarly to Affinity Clustering and k-Means, outperforming many state-of-the-art serial algorithms. Along the way, we also introduce an efficient k-sized maximum matching algorithm in the MPC model."

Matching Affinity Clustering: Improved Hierarchical Clustering at Scale with Guarantees

Option-Critic in Cooperative Multi-agent Systems

Researchers have long been interested in the role that norms can play in governing agent actions in multi-agent systems. Much work has been done on formalising normative concepts from human society and adapting them for the government of open software systems, and on the simulation of normative processes in human and artificial societies. However, there has been comparatively little work on applying normative MAS mechanisms to understanding the norms in human society. This work investigates this issue in the context of international politics. Using the GDELT dataset, containing machine-encoded records of international events extracted from news reports, we extracted bilateral sequences of inter-country events and applied a Bayesian norm mining mechanism to identify norms that best explained the observed behaviour. A statistical evaluation showed that the normative model fitted the data significantly better than a probabilistic discrete event model.

Mining International Political Norms from the GDELT Database

Designing Truthful Contexual Multi-Armed Bandits based Sponsored Search Auctions

Reinforcement learning algorithms have been shown to converge to the classic replicator dynamics of evolutionary game theory, which describe the evolutionary process in the limit of an infinite population. However, it is not clear how to interpret these dynamics from the perspective of a learning agent. In this paper we propose a data-inefficient batch-learning algorithm for temporal difference Q learning and show that it converges to a recently proposed deterministic limit of temporal difference reinforcement learning. In a second step, we state a data-efficient learning algorithm, that uses a form of experience replay, and show that it retains core features of the batch learning algorithm. Thus, we propose an agent-interpretation for the learning dynamics: What is the infinite population limit of evolutionary dynamics is the infinite memory limit of learning dynamics.

Downloads

Next from AAMAS 2020

Silly rules improve the capacity of agents to learn stable enforcement and compliance behaviors

Similar lecture

State-Conditioned Adversarial Subgoal Generation

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Silly rules improve the capacity of agents to learn stable enforcement and compliance behaviors

Similar lecture

State-Conditioned Adversarial Subgoal Generation

Downloads