Kunikazu Kobayashi, Koji Nakano, Takashi Kuremoto and Masanao Obayashi (February 1st 2010). Objective-based Reinforcement Learning System for Cooperative Behavior Acquisition, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8615. 

Daniela P. Alves, Li Weigang and Bueno B. Souza (January 1st 2008). Reinforcement Learning to Support Meta-Level Control in Air Traffic Management, Reinforcement Learning, Cornelius Weber, Mark Elshaw and Norbert Michael Mayer, IntechOpen, DOI: 10.5772/5293. 

Brendon Patrick Hyndman and Shirley Wyver (April 2nd 2020). Outdoor Recreation within the School Setting: A Physiological and Psychological Exploration [Online First], IntechOpen, DOI: 10.5772/intechopen.91900. 

New Zealand

AAMAS 2020

Off Policy Deep Reinforcement Learning with Analogous Disentangled Exploration

**31st IEEE International Conference on Robot & Human Interactive Communication**
 This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.
 
 The conference covers a wide range of topics related to Robot and Human Interactive Communication, involving theories, methodologies, technologies, empirical and experimental studies. Papers related to the study of robotic technology, psychology, cognitive science, artificial intelligence, human factors, ethics and policies, interaction-based robot design and other topics related to human-robot interaction are welcome.
 
 **CONFERENCE THEME:**
 Social, asocial, and antisocial robots
 
 Robots are starting to populate our lives: at work, in our homes, hotels, shops. However, not everybody is comfortable with robots finding a place in society. For example, people may be afraid that robots will replace humans in their jobs. This belief is exacerbated by the media that, on one side, raise expectations of robots’ capabilities, while amplifying fears and worries on the other side.
 
 In the next years, due to the increasing request for interaction with people in social environments, academic research and market analyses predict that robots will become more and more “social”. But, at the same time, robots may incur the risk of being perceived as “asocial” as their pervasiveness in everyday environments may become an obstacle to social interaction between humans. Or they may even be rejected as “antisocial” if researchers do not take care to develop robots that behave according to the customs and habits of the communities where they are expected to operate.
 
 To prevent this from happening, robotic researchers can no longer take the luxury of being mere technicians. They must be capable of addressing all the challenges raised by future robot-populated communities in a multidisciplinary way. They need to share their knowledge with the public to make people aware of what robots really are, what expectations we can have from them, the risks we may incur, and the possible mitigation plans.
 
 While being open to any kind of contribution related to Human-Robot Interactive Communication, the RO-MAN 2022 conference will emphasize these aspects of robotic research. How can we create robots capable of being welcomed as our assistants and companions in society while, at the same time, not being perceived as “asocial” or “antisocial?” How can we create robots perceived as facilitators of the natural development of human beings and communities, rather than obstacles?
 
 Contributions from different disciplines, not exclusively related to technological development, will be welcomed as crucial to pave the way to global acceptance of future robots with an informed attitude.
 
 [![](https://assets.underline.io/uploads/markdown_image/1/image/af259abe105b3791c498ae1552d75710.png)](https://drive.google.com/file/d/18w73BZPB_JVOgcrcdZDqYpEupwVt4CnU/view?usp=sharing)

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

[Register Here](http://www.smile.unina.it/ro-man2022/welcome-to-ieee-ro-man-2022/)

Registration Is Required

IEEE RO-MAN 2022

31st IEEE International Conference on Robot & Human Interactive Communication
This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Reinforcement learning (RL), like any on-line learning method, inevitably faces the exploration-exploitation dilemma. When a learning algorithm requires as few data samples as possible, it is called sample efficient. The design of sample-efficient algorithms is an important area of research. Interestingly, all currently known provably efficient model-free RL algorithms utilize the same well-known principle of optimism in the face of uncertainty. We unite these existing algorithms into a single general model-free optimistic RL framework. We show how this facilitates the design of new optimistic model-free RL algorithms by simplifying the analysis of their efficiency. Finally, we propose one such new algorithm and demonstrate its performance in an experimental study.

Generalized Optimistic Q-Learning with Provable Efficiency

A patrol of robot teams, where the robots are required to repeatedly visit a target area, is a useful tool in detecting an adversary trying to penetrate. In this work we examine the Closed Perimeter Patrol problem, in which the robots travel along a closed perimeter and the adversary is aware of the robots' patrol policy. The goal is to maximize the probability of penetration detection. Previous work dealt with symmetric tracks, in which all parts of the track have similar properties, and suggested non-deterministic patrol schemes, characterized by a uniform policy along the entire area. We consider more realistic scenarios of asymmetric tracks, with various parts of the track having different properties, and suggest a patrol policy with a non-uniform policy along different points of the track. We compare the achievements of both models and show the advantage of the non-uniform model. We further explore methods to efficiently calculate the attributes needed to maximize the probability of penetration detection and compare their implementation in various scenarios.


Non-Uniform Policies for Multi-Robot Asymmetric Perimeter Patrol in Adversarial Domains

Multi-Agent Path Finding (MAPF) plays an important role in many real-life applications where autonomous agents must coordinate to reach their goals without collisions. MAPF problems often take place in structured environments that are usually assumed to be static and known in advance. In this paper, we introduce C-MAPF, i.e., MAPF in Configurable environments, a novel variant of the MAPF problem in which the environment is configurable, namely its structure and topology can be controlled within some given constraints. Consider, for instance, a warehouse logistics application: the environment can be changed (at least to some degree) by the managers of the warehouse, for example by re-arranging the positions of the shelves or by removing or adding temporary walls. We study the properties of the C-MAPF problem and we devise two algorithms for solving it, both based on Conflict-Based Search (CBS), a state-of-the-art MAPF algorithm. First, we present Parallel CBS (P-CBS), that searches for a solution by simultaneously considering all the possible configurations of the environment. We then present Abstract CBS (A-CBS), an extended version of the CBS algorithm that solves C-MAPF problems by introducing a new type of conflict on the allowable configurations of the environment. We prove that our solvers are both complete and optimal and we experimentally assess their performance in different settings.


Multi-Agent Path Finding in Configurable Environments

In many settings where multiple agents interact, the optimal choices for each agent depend heavily on the choices of the others.
These coupled interactions are well-described by a general-sum differential game, in which players have differing objectives, the state evolves in continuous time, and optimal play may be characterized by one of many equilibrium concepts, e.g., a Nash equilibrium.
Often, problems admit multiple equilibria.
From the perspective of a single agent in such a game, this multiplicity of solutions can introduce uncertainty about how other agents will behave.
This paper proposes a general framework for resolving ambiguity between equilibria by reasoning about the equilibrium other agents are aiming for.
We demonstrate this framework in simulations of a multi-player human-robot navigation problem that yields two main conclusions:
First, by inferring which equilibrium humans are operating at, the robot is able to predict trajectories more accurately, and second, by discovering and aligning itself to this equilibrium the robot is able to reduce the cost for all players.


Inference-Based Strategy Alignment for General-Sum Differential Games

Multiagent teams have been shown to be effective in many domains that require coordination among team members. However, finding valuable joint-actions becomes increasingly difficult in tightly-coupled domains where each agent's performance depends on the actions of many other agents. Reward shaping partially addresses this challenge by deriving more "tuned" rewards to provide agents with additional feedback, but this approach still relies on agents randomly discovering suitable joint-actions. In this work, we introduce Counterfactual Agent Suggestions (CAS) as a method for injecting knowledge into an agent's learning process within the confines of existing reward structures. We show that CAS enables agent teams to converge towards desired behaviors more reliably. We also show that improvement in team performance in the presence of suggestions extends to large teams and tightly-coupled domains.


The Power of Suggestion

Reinforcement learning systems require good representations to work well. For decades practical success in reinforcement learning was limited to small domains. Deep reinforcement learning systems, on the other hand, are scalable, not dependent on domain specific prior knowledge and have been successfully used to play Atari, in 3D navigation from pixels, and to control high degree of freedom robots. Unfortunately, the performance of deep reinforcement learning systems is sensitive to hyper-parameter settings and architecture choices. Even well tuned systems exhibit significant instability both within a trial and across experiment replications. In practice, significant expertise and trial and error are usually required to achieve good performance. One potential source of the problem is known as catastrophic interference: when later training decreases performance by overriding previous learning. Interestingly, the powerful generalization that makes Neural Networks (NN) so effective in batch supervised learning might explain the challenges when applying them in reinforcement learning tasks. In this paper, we explore how online NN training and interference interact in reinforcement learning. We find that simply re-mapping the input observations to a high-dimensional space improves learning speed and parameter sensitivity. We also show this preprocessing reduces interference in prediction tasks. More practically, we provide a simple approach to NN training that is easy to implement, and requires little additional computation. We demonstrate that our approach improves performance in both prediction and control with an extensive batch of experiments in classic control domains.

Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks

Toll-Based Learning for Minimising Congestion under Heterogeneus Preferences

In multiagent problems that require complex joint actions, reward shaping methods yield good behavior by incentivizing the agents' potentially valuable actions. However, reward shaping often requires access to the functional form of the reward function and the global state of the system. In this work, we introduce the Exploratory Gaussian Reward (EGR), a new reward model that creates optimistic stepping stone rewards linking the agents potentially good actions to the desired joint action. EGR models the system reward as a Gaussian Process to leverage the inherent uncertainty in reward estimates that push agents to explore unobserved state space. In the tightly coupled rover coordination problem, we show that EGR significantly outperforms a neural network approximation baseline and is comparable to the system with access to the functional form of the global reward. Finally, we demonstrate how EGR improves performance over other reward shaping methods by forcing agents to explore and escape local optima.

Gaussian Processes as Multiagent Reward Models

With the tremendous development of AI technologies, people will increasingly encounter software algorithms that supervise their work. Algorithmic management is the term for AI that performs the functions traditionally reserved for human managers (hiring, firing, providing evaluative feedback, and setting compensation). Although such algorithms indisputably perform management functions, they are often framed as support tools that facilitate worker autonomy. Perceptions of autonomy can enhance productivity, especially when the work holds intrinsic meaning for workers. But crowdwork often seems meaningless. More problematically, the meaning of the work must sometimes be obscured due to reasons of security or experimental control (when the workers serve as subjects in a psychological experiment). In this paper, we conduct an online experiment (N=560) to investigate how autonomy-perceptions and the meaningfulness of work interact to shape crowdworker motivation. As predicted, we find that workers are motivated when their work has meaning and algorithmic management is framed in a way that makes worker autonomy salient. However, when work holds no meaning, we find productivity is enhanced when algorithms are framed in a way that makes algorithm control salient. We also find evidence that providing meaning to the work can introduce systematic biases in crowdworker responses that could undermine accuracy in certain contexts.


The Effects of Autonomy and Task meaning in Algorithmic Management of Crowdwork

Agents can individually devise plans and coordinate to achieve common goals. Methods exist to factor planning problems into separate tasks and distribute the plan synthesis process, while reducing the overall planning complexity. However, merging distributedly generated plans becomes computationally costly when task plans are tightly coupled, and conflicts arise due to dependencies between plan actions. Existing methods either scale poorly as the number of agents and tasks increases, or do not minimize makespan, the overall time necessary to execute all tasks. A new algorithm, the Temporal Optimal Conflict Resolution Algorithm (TCRA*), is introduced to merge independently generated plans and optimally minimize the resulting makespan. A proof of optimality is provided and the algorithm is empirically evaluated across two heterogeneous multiagent domains against two baseline algorithms. The TCRA* results in better makespan across the problems solved, and a search relaxation constant allows the TCRA* to generate better plans with competitive processing time and memory usage. 


Off Policy Deep Reinforcement Learning with Analogous Disentangled Exploration

Downloads

Next from AAMAS 2020

Generalized Optimistic Q-Learning with Provable Efficiency

Similar lecture

Expression of Personality by Gaze Movements of an Android Robot in Multi-Party Dialogues

Off Policy Deep Reinforcement Learning with Analogous Disentangled Exploration

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Generalized Optimistic Q-Learning with Provable Efficiency

Similar lecture

Expression of Personality by Gaze Movements of an Android Robot in Multi-Party Dialogues

Downloads