Kunikazu Kobayashi, Koji Nakano, Takashi Kuremoto and Masanao Obayashi (February 1st 2010). Objective-based Reinforcement Learning System for Cooperative Behavior Acquisition, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8615. 

Daniela P. Alves, Li Weigang and Bueno B. Souza (January 1st 2008). Reinforcement Learning to Support Meta-Level Control in Air Traffic Management, Reinforcement Learning, Cornelius Weber, Mark Elshaw and Norbert Michael Mayer, IntechOpen, DOI: 10.5772/5293. 

New Zealand

Existing Deep Reinforcement Learning (DRL) methods are still faced with sample inefficiency, so it is difficult to learn from scratch. Transfer Learning (TL) has shown great potential to accelerate DRL by leveraging prior knowledge from past learned policies of relevant tasks. Existing transfer approaches either explicitly computes the similarity between tasks or select appropriate source policies to provide guided explorations for the target task. However, how to directly optimize the target policy by alternatively utilizing knowledge from appropriate source policies without explicitly measuring the similarity is currently missing. We propose a novel Policy Transfer Framework (PTF) based on this idea. Our PTF models multi-policy transfer as option learning, and we directly optimize the target policy using the transferred knowledge. We propose an adaptive, heuristic mechanism to avoid negative transfer. PTF can be easily combined with DRL methods. And experimental results show it outperforms existing DRL approaches, and state-of-the-art policy transfer methods both in discrete and continuous action space.


AAMAS 2020

Efficient Deep Reinforcement Learning through Policy Transfer

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Social structures naturally arise from social networks, yet no model well interprets the emergence of structural properties in a unified dimension. Here, we unify explanations for the emergence of net- work structures by revealing the pivotal role of social capital, i.e., benefits that a society grants to individuals, in network formation. We propose a game-based framework social capital games that math- ematically conceptualizes social capital. Through this framework, individuals are regarded as independent learning agents that aim to gain social capital via building interpersonal ties. We adopt multi- agent reinforcement learning (MARL) to train agents. By varying configurations of the game, we observe the emergence of classical structures of community, small-world, and core-periphery.

Social Structure Emergence: A multi-agent Reinforcement Learning Framework for Relationship Building

Realistic models of decision-making and social interactions, considering the nature of memory and biases, continue to be an area of immense interest. Emotion and mood are a couple of key factors that play a major role in decisions, nature of social interactions, size of the social network, and the level of engagement. Most of the prior work in this direction focused on a single trait, behavior, or bias. However, this work builds an integrated model that considers multiple traits such as loneliness, the drive to interact, the memory, and mood biases in an agent. The agent system comprises of rational, manic, depressed, and bipolar agents. The system is modeled with an interconnected network, and the size of the personal network of each agent is based on its nature. We consider a game of iterated interactions where an agent cooperates based on its past experiences with the other agent. Through simulation, the effects of various biases and comparative performances of agent types is analyzed. Taking the performance of rational agents as the baseline, manic agents do much better, and depressed agents do much worse. The payoffs also exhibit an almost-linear relationship with the extent of mania.


Analyzing the Effects of Memory Biases and Mood Disorders on Social Performance

The study of stable fractional matchings is fairly recent and moreover, is scarce. This paper reports the first investigation into the important but unexplored topic of incentive compatibility of matching mechanisms to find stable fractional matchings.We focus our attention on matching instances under strict preferences. First, we make the significant observation that there are matching instances for which no mechanism that produces a stable fractional matching is incentive compatible.We then characterize restricted settings of matching instances admitting unique stable fractional matchings. For this class of instances, we prove that every mechanism that produces the unique stable fractional matching is (a) incentive compatible and (b) resistant to coalitional manipulations. We provide a polynomial-time algorithm to compute the stable fractional matching as well.

A Study of Incentive Compatibility and Stability Issues in Fractional Matchings

"The theme park problem is a research framework that evaluates measures for improving the satisfaction of visitors to crowded amusement parks on a multi-agent simulation (MAS).
To make the MAS more realistic, we propose the followings:
1) visitor surplus, which evaluates visitors' satisfaction based on microeconomics,
2) multinomial linear model, a selection behavior model based on visitor surplus,
and 3) a tolerance limit model, which estimates the distribution of the visitors' tolerance limits of waiting times by analyzing questionnaire results."


Theme Park Simulation based on Questionnaires for Maximizing Visitor Surplus

We present a method that analyzes a person's negotiation behavior to automatically detect co-occurrence of tactics and combination of tactics (i.e., negotiation styles). We first identify action features consistent with use of the common negotiation tactics based on prior research in negotiation. Next, we apply regularized linear regression over a negotiation dataset to assess how effective particular tactics are in predicting the negotiation outcome. Finally, we use a supervised variant of a topic model to derive effective negotiation styles. Results from the clusters produced by the topic models provide insights regarding the effectiveness of negotiation styles that people utilize.


A Supervised Topic Model Approach to Learning Effective Styles within Human-Agent Negotiation

The problem of truth discovery, i.e., of trying to find the true facts concerning a number of objects based on reports from various information sources of unknown trustworthiness, has received increased attention recently. The problem is made interesting by the fact that the relative believability of facts depends on the trust- worthiness of their sources, which in turn depends on the believability of the facts the sources report. Several algorithms for truth discovery have been proposed, but their evaluation has mainly been performed experimentally by computing accuracy against large datasets. Furthermore, it is often unclear how these algorithms behave on an intuitive level. In this talk I outline our approach to a general framework for truth discovery, which allows comparison and evaluation of algorithms based instead on their theoretical properties. To do so we pose truth discovery as a social choice problem, and formulate various axioms that any reasonable algorithm should satisfy.


An Axiomatic Approach to Truth Discovery

"The Multi-Robot Pickup and Delivery problem has received significant attention from the research community.
We focus on such problem and propose a multi-item variant, where robots can pick up multiple items at once and deliver them to their destinations in a single travel.
We propose a distributed algorithm based on a Token-Passing approach, where robots handles by themselves the allocation of tasks and generates conflict-free paths requiring weaker assumptions in comparison to previous approaches.
We evaluate our approach on two maps, one representing the ICE laboratory, a facility for Industry 4.0 located in Verona.
Our approach produces solutions comparable in quality to those produced by a centralized approach and reduces computation times significantly.
We also show a synchronization technique to the robots' movements, still based on Token-Passing. This guarantees the absence of collisions in spite of unexpected delays in path execution."


Decentralized Task Assignment for Multi-Item Pickup and Delivery in Logistic Scenarios

In many collective decision making situations, agents use voting to choose an alternative that best represents the preferences of the group. It is often assumed that voters will vote truthfully rather than expending the effort needed to manipulate the outcome in cognitively and computationally complex situations. However, being truthful is just one possible heuristic that agents may employ. We examine how real voters employ heuristics in a variety of approval voting scenarios. In particular, we consider heuristics where a voter ignores information about other voting profiles and makes their decisions based solely on how much they like each candidate. In a behavioral experiment, we show that people vote truthfully in some situations, but prioritize high utility candidates in others. We show how the structure of the voting environment affects how well each heuristic performs as well as how and when humans employ these different heuristics.


Heuristic Strategies in Uncertain Approval Voting Environments

An underlying assumption of Stackelberg Games (SGs) is perfect rationality of the players. However, in real-life situations the followers (thieves, poachers, smugglers), as humans in general, may act not in a perfectly rational way, since their decisions may be affected by biases of various kinds which bound rationality of their decisions. One of the popular models of bounded rationality is Anchoring Theory (AT) which claims that humans have a tendency to flatten probabilities of available options, i.e. they perceive a distribution of these probabilities as being closer to the uniform distribution than it really is. We propose an efficient formulation of AT in sequential extensive-form SGs suitable for Mixed-Integer Linear Program solution methods and compare the results of its implementation in five state-of-the-art methods for solving sequential SGs.


Anchoring Theory in Sequential Stackelberg Games

Online advertising campaigns are typically launched for a customer across multiple touch points (scenarios) before the conversion of his final purchase. To maximize the advertisers’ revenue, it requires the platform to develop its advertising strategy based on the consumers’ behavioral trajectories in the previous scenarios. Meanwhile, it is also critical to maintain the interpretability of the models on the conversion rate; however, modern reinforcement learning based solutions fail to do so due to their black-box modeling on the consumer intents. In this paper, we model consumer’s purchase intention as a latent variable, and formulate the advertising problem as a partially observed Markov Decision Process (POMDP). We incorporate the expectation maximization (EM) algorithms for solving the optimal POMDP. Our extensive experiments based on large-scale real-world data demonstrate that our method provides superior performance over several baselines. Apart from the improved advertising performance, our method is able to offer interpretation on the attribution of the conversion.


Efficient Deep Reinforcement Learning through Policy Transfer

Downloads

Next from AAMAS 2020

Social Structure Emergence: A multi-agent Reinforcement Learning Framework for Relationship Building

Similar lecture

Exploring Temporal Information Dynamics in Spiking Neural Networks

Efficient Deep Reinforcement Learning through Policy Transfer

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Social Structure Emergence: A multi-agent Reinforcement Learning Framework for Relationship Building

Similar lecture

Exploring Temporal Information Dynamics in Spiking Neural Networks

Downloads