Toshihiko Watanabe (February 1st 2010). Hierarchical Reinforcement Learning Using a Modular Fuzzy Model for Multi-Agent Problem, New Advances in Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/9388. 

Masanao Obayashi, Kenichiro Narita, Yohei Okamoto, Takashi Kuremoto, Kunikazu Kobayashi and Liangbing Feng (January 14th 2011). A Reinforcement Learning System Embedded Agent with Neural Network-Based Adaptive Hierarchical Memory Structure, Advances in Reinforcement Learning, Abdelhamid Mellouk, IntechOpen, DOI: 10.5772/13199.

Le Tien Dung, Takashi Komeda and Motoki Takagi (January 1st 2009). Solving POMDPs with Automatic Discovery of Subgoals, Theory and Novel Applications of Machine Learning, Meng Joo Er and Yi Zhou, IntechOpen, DOI: 10.5772/6685. 

New Zealand

Hierarchical Reinforcement Learning (HRL) is a promising approach to solve more complex tasks which may be challenging for the traditional reinforcement learning. HRL achieves this by decomposing a task into shorter-horizon subgoals as targets to achieve. Autonomous discovery of such subgoals is an important part of HRL. Recently, end-to-end HRL methods have been used to reduce the overhead from offline subgoal discovery by seeking the useful subgoals while simultaneously learning optimal policies in a hierarchy. However, these methods may still suffer from slow learning when the search space used by a high level policy to find the subgoals is large. We propose LIDOSS, an end-to-end HRL method with an integrated heuristic for subgoal discovery, to address this problem.

AAMAS 2020

Hierarchical Reinforcement Learning with Integrated Discovery of Salient Subgoals

 **RTSS 2021** has successfully concluded. Hope you have enjoyed it, and look forward to seeing you next year in person.
    Announcement for the Awards:
 * Best Paper Award goes to: Monte Carlo Response-Time Analysis, Sergey Bozhko, Georg von der Brüggen, and Björn Brandenburg
 * Best Student Paper Award goes to: Automatic Energy-Hotspot Detection and Elimination in Real-Time Deeply Embedded Systems, Mohsen Shekarisaz, Lothar Thiele, and Mehdi Kargahi.
 * Best Solution Award for Industry Session goes to: Toward Real-Time Guaranteed Scheduling for Autonomous Driving Systems. Jinghao Sun, Tianyi Wang, Kailu Duan, Bin Lu, Jiankang Ren, Zhishan Guo, and Guozhen Tan.



IEEE RTSS 2021

The IEEE Real-Time Systems Symposium (RTSS) is the premier conference in the field of real-time systems, and is a venue for researchers and practitioners to showcase innovations covering all aspects of real-time systems, including theory, design, analysis, implementation, evaluation, and experience. 

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Silly rules improve the capacity of agents to learn stable enforcement and compliance behaviors

This paper studies the complexity of two microbribery problems under the model of group identification. In these problems, we are given a subset of distinguished individuals, and the questions are whether these individuals can be made socially qualified or whether they can be made exactly the socially qualified individuals, respectively, by modifying a limited number of entries in the qualifications-profile. For consent rules, the consensus-start-respecting rule, and the liberal-start-respecting rule, we obtain many $\np$-hardness results as well as polynomial-time solvability results. We also study the problems in $r$-profiles where each individual qualifies exactly $r$ individuals.

Microbribery in Group Identification

Multi-Agent Path Finding (MAPF) is the problem of moving a team of agents from their start locations to their goal locations without collisions. We study the lifelong variant of MAPF where agents are constantly engaged with new goal locations, such as in warehouses. We propose a new framework for solving lifelong MAPF by decomposing the problem into a sequence of Windowed MAPF instances, where a Windowed MAPF solver resolves collisions among the paths of agents only within a finite time horizon and ignores collisions beyond it. Our framework is particularly well suited to generating pliable plans that adapt to continually arriving new goal locations. We evaluate our framework with a variety of MAPF solvers and show that it can produce high-quality solutions for up to 1,000 agents, significantly outperforming existing methods.

Lifelong Multi-Agent Path Finding in Large-Scale Warehouses

"Hierarchical clustering is a stronger extension of one of today's most influential unsupervised learning methods: clustering. The goal of this method is to create a hierarchy of clusters, thus constructing cluster evolutionary history and simultaneously finding clusterings at all resolutions. We propose four traits of interest for hierarchical clustering algorithms: (1) empirical performance, (2) theoretical guarantees, (3) balance (the minimum ratio between cluster sizes), and (4) scalability. While a number of algorithms are designed to achieve one to two of these traits at a time, there exist none that achieve all four.



Inspired by Bateni et al.'s scalable and empirically successful Affinity Clustering [NeurIPs 2017], we introduce Affinity's successor, Matching Affinity Clustering. Like its predecessor, Matching Affinity Clustering maintains strong empirical performance, even outperforming Affinity when the dataset is size 2^n and clusters are balanced, and uses Massively Parallel Communication as its distributed model. Designed to maintain provably balanced clusters, we show that our algorithm achieves a (1/3-epsilon)-approximation for Moseley and Wang's revenue (the dual to Dasgupta's cost) when the data set is of size 2^n, and a (1/9-epsilon)-approximation in general. We prove the former approximation is tight, and also that Affinity Clustering cannot do better than a 1/O(n)-approximation. In addition, we see that our algorithm empirically performs similarly to Affinity Clustering and k-Means, outperforming many state-of-the-art serial algorithms. Along the way, we also introduce an efficient k-sized maximum matching algorithm in the MPC model."

Matching Affinity Clustering: Improved Hierarchical Clustering at Scale with Guarantees

Option-Critic in Cooperative Multi-agent Systems

Researchers have long been interested in the role that norms can play in governing agent actions in multi-agent systems. Much work has been done on formalising normative concepts from human society and adapting them for the government of open software systems, and on the simulation of normative processes in human and artificial societies. However, there has been comparatively little work on applying normative MAS mechanisms to understanding the norms in human society. This work investigates this issue in the context of international politics. Using the GDELT dataset, containing machine-encoded records of international events extracted from news reports, we extracted bilateral sequences of inter-country events and applied a Bayesian norm mining mechanism to identify norms that best explained the observed behaviour. A statistical evaluation showed that the normative model fitted the data significantly better than a probabilistic discrete event model.

Mining International Political Norms from the GDELT Database

Designing Truthful Contexual Multi-Armed Bandits based Sponsored Search Auctions

Reinforcement learning algorithms have been shown to converge to the classic replicator dynamics of evolutionary game theory, which describe the evolutionary process in the limit of an infinite population. However, it is not clear how to interpret these dynamics from the perspective of a learning agent. In this paper we propose a data-inefficient batch-learning algorithm for temporal difference Q learning and show that it converges to a recently proposed deterministic limit of temporal difference reinforcement learning. In a second step, we state a data-efficient learning algorithm, that uses a form of experience replay, and show that it retains core features of the batch learning algorithm. Thus, we propose an agent-interpretation for the learning dynamics: What is the infinite population limit of evolutionary dynamics is the infinite memory limit of learning dynamics.

Hierarchical Reinforcement Learning with Integrated Discovery of Salient Subgoals

Downloads

Next from AAMAS 2020

Silly rules improve the capacity of agents to learn stable enforcement and compliance behaviors

Similar lecture

AI Meets Real-Time: Addressing Real-World Complexities in Graph Response-Time Analysis

Hierarchical Reinforcement Learning with Integrated Discovery of Salient Subgoals

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Silly rules improve the capacity of agents to learn stable enforcement and compliance behaviors

Similar lecture

AI Meets Real-Time: Addressing Real-World Complexities in Graph Response-Time Analysis

Downloads