New Zealand


Continuous Action-space Interactive Reinforcement learning (CAIR) is a continuous action-space interactive reinforcement learning algorithm that can out-preform state-of-the-art reinforcement learning algorithms early on in training. We test CAIR in two simulated robotics environments with intuitive and easy to design heuristic teachers.

AAMAS 2022

Environment Guided Interactive Reinforcement Learning: Learning from Binary Feedback in High-Dimensional Robot Task Environments

interactive reinforcement learning

human robot interaction

reinforcement learning

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. This theme also touches upon people's expectations and perceptions of robots, as well as other social, cognitive and cultural factors that will influence human-robot interactions. We will have a special session/track on this topic, invite targeted submissions and include plenary speakers who are specialist in this area. We sincerely hope that this focus will help enlighten the design-thinking of all of us as we fabricate and program our own various systems.

To access RO-MAN Virtual 2021, [please register here.](https://ro-man2021.org/registration/)

IEEE RO-MAN 2021

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. 

poster

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).
<br>
<br>



A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

Recently, text world games have been proposed to enable artifi- cial agents to understand and reason about real-world scenarios. These text-based games are challenging for artificial agents, as it requires an understanding of and interaction using natural language in a partially observable environment. Agents observe the environment via textual descriptions designed to be challenging enough for even human players. Past approaches have not paid enough attention to the language understanding capability of the proposed agents. Typically, these approaches train from scratch, an agent that learns both textual representations and the gameplay online during training using a temporal loss function. Given the sample-inefficiency of RL approaches, it is inefficient to learn rich enough textual representations to be able to understand and reason using the textual observation in such a complicated game environment setting. In this paper, we improve the semantic understanding of the agent by proposing a simple RL with LM framework where we use transformer-based language models with Deep RL models. We perform a detailed study of our framework to demonstrate how our model outperforms all existing agents on the popular game, Zork1, to achieve a score of 44.7, which is 1.6 higher than the state-of-the- art model. Overall, our proposed approach outperforms 4 games out of the 14 text-based games, while performing comparable to the state-of-the-art models on the remaining games.

Pre-trained Language Models as Prior Knowledge for Playing Text-based Games


Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world. Multi-agent reinforcement learning (MARL) has the potential to achieve this goal, demonstrating success in a series of challenging problems. However, whilst these advances are significant, the vast majority of focus has been on the self-play paradigm. This often results in a coordination problem, caused by agents learning to make use of arbitrary conventions when playing with themselves. This means that even the strongest self-play agents may have very low cross-play with other agents, including other initializations of the same algorithm. In this paper we propose to solve this problem by adapting agent strategies on the fly, using a posterior belief over the other agents' strategy. Concretely, we consider the problem of selecting a strategy from a finite set of previously trained agents, to play with an unknown partner. We propose an extension of the classic statistical technique, Gibbs sampling, to update beliefs about other agents and obtain close to optimal ad-hoc performance. Despite its simplicity, our method is able to achieve strong cross-play with unseen partners in the challenging card game of Hanabi, achieving successful ad-hoc coordination without knowledge of the partner's strategy a priori.


On-the-fly Strategy Adaptation for ad-hoc Agent Coordination


Many real-world applications require an agent to make robust and deliberate decisions with multimodal information (e.g., robots with multi-sensory inputs). However, it is very challenging to train the agent via reinforcement learning (RL) due to the heterogeneity and dynamic importance of different modalities. Specifically, we observe that these issues make conventional RL methods difficult to learn a useful state representation in the end-to-end training with multimodal information. To address this, we propose a novel multimodal RL approach that can do multimodal alignment and importance enhancement according to their similarity and importance in terms of RL tasks respectively. By doing so, we are able to learn an effective state representation and consequentially improve the RL training process. We test our approach on several multimodal RL domains, showing that it outperforms state-of-the-art methods in terms of learning speed and policy quality.

Multimodal Reinforcement Learning with Effective State Representation Learning


Augmented reality (AR) technologies have been applied to human-robot collaboration (HRC) domains to enable people to visualize the state of the robots.
Current AR-based visualization strategies are manually designed. This design process requires a lot of human efforts, and domain knowledge. When too little information is visualized, human users find the AR interface not useful; when too much is visualized, they find it difficult to process the visualized information. In this paper, we develop an intelligent AR agent that learns visualization policies (what to visualize, when, and how) from demonstrations. We developed a Unity-based platform for simulating warehouse environments where human-robot teammates work on collaborative delivery tasks. We have collected a dataset that includes 6000 demonstrations of visualizing robots' current and planned behaviors. Our results from experiments with real human participants show that, compared with competitive baselines from the literature, our learned visualization strategy significantly increases the efficiency of human-robot teams in delivery tasks.

Learning Visualization Policies of Augmented Reality for Human-Robot Collaboration

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1744.pdf

**Abstract:**



Autonomous Flight Arcade Challenge: Single- and Multi-Agent Learning Environments for Aerial Vehicles


When multiple humans and robots are moving in spaces like restaurants, hospitals, or banks, making the robot's movements easy to predict can help the humans co-navigate the space with the robots. Since people would be busy with their own goals, they are not paying close attention to the prior movements, or goals of multiple robots. So predictability from the robot's current position alone would help. With this in mind, we propose using an algorithm to lay out fixed paths for the different tasks the robots would do, such that predictability from only the current position alone is optimized, and motion costs are kept within acceptable bounds.

Minimizing Robot Navigation Graph For Position-Based Predictability By Humans


In network science, centrality indices are used to determine important nodes by virtue of their position in a network.
In social choice theory, voting rules are used to aggregate preferences of voters to determine the winners of an election.
Exploiting parallels between these two fields, we propose a novel approach to define network centrality indices based on voting rules.
Since formal properties of voting rules have been studied in much greater depth, this will not only lead to new applications of social choice theory but also facilitate a deeper understanding of centrality in networks.

Voting for Centrality


We study the problem of finding fair and efficient allocations of a set of indivisible items to a set of agents, where each item may be a good (positively valued) for some agents and a bad (negatively valued) for others, i.e., a mixed manna. As fairness notions, we consider arguably the strongest possible relaxations of envy-freeness and proportionality, namely envy-free up to any item (EFX and EFX_0), and proportional up to the maximin good or any bad (PropMX and PropMX_0). Our efficiency notion is Pareto-optimality (PO).

We study two types of instances: (i) Separable, where the item set can be partitioned into goods and bads, and (ii) Restricted mixed goods (RMG), where for each item j, every agent has either a non-positive value for j, or values j at the same v_j > 0. We obtain polynomial-time algorithms for the following:

* Separable instances: PropMX_0 allocation.
* RMG instances: Let pure bads be the set of items that everyone values negatively.
- PropMX allocation for general pure bads
- EFX+PropMX allocation for identically-ordered pure bads
- EFX+PropMX+PO allocation for identical pure bads.

Finally, if the RMG instances are further restricted to binary mixed goods where all the v_j's are the same, we strengthen the results to guarantee EFX_0 and PropMX_0 respectively.

(Almost) Envy-Free, Proportional and Efficient Allocations of an Indivisible Mixed Manna


Moving target defense (MTD) has emerged as a key technique that can be used in various security applications to reduce the threat of attackers by taking away their ability to perform reconnaissance and exploit vulnerabilities. However, most of the existing research in the field assumes unrealistic access to information about the attacker's motivations and/or actions when developing MTD strategies. Many of the existing approaches also assume complete knowledge regarding the vulnerabilities of a particular application and how each of these vulnerabilities can be exploited by an attacker. In this work, we propose an algorithm that generates effective MTD strategies for web applications that does not rely on prior knowledge about the attackers. Our approach assumes that the only information the defender receives about its own reward function, is via interaction with the attacker in a repeated game setting. We evaluate our algorithm using data which is mined from the National Vulnerability Database to show that it matches the performance of the state of the art techniques, despite using much less information.

Moving Target Defense under Uncertainty for Web Applications


For cooperative mutliagent reinforcement learning tasks, we propose a novel value factorization framework in the popular centralized training with decentralized execution paradigm, called multiagent Q-learning with sub-team coordination (QSCAN). This framework could flexibly exploit local coordination within sub-teams for effective factorization while honoring the individual-global-max (IGM) condition.
QSCAN encompasses the full spectrum of sub-team coordination according to sub-team size, ranging from the monotonic value function class to the entire IGM function class, with familiar methods such as QMIX and QPLEX located at the respective extremes of the spectrum. Empirical results show that QSCAN's performance dominates state-of-the-art methods in predator-prey tasks and the Switch challenge in MA-Gym.

Environment Guided Interactive Reinforcement Learning: Learning from Binary Feedback in High-Dimensional Robot Task Environments

Downloads

Next from AAMAS 2022

Pre-trained Language Models as Prior Knowledge for Playing Text-based Games

Similar lecture

Social norms and cooperation in a collective-risk social dilemma:comparing reinforcement learning and norm-based approaches

Environment Guided Interactive Reinforcement Learning: Learning from Binary Feedback in High-Dimensional Robot Task Environments

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

Pre-trained Language Models as Prior Knowledge for Playing Text-based Games

Similar lecture

Social norms and cooperation in a collective-risk social dilemma:comparing reinforcement learning and norm-based approaches

Downloads