New Zealand


Many real-world applications require an agent to make robust and deliberate decisions with multimodal information (e.g., robots with multi-sensory inputs). However, it is very challenging to train the agent via reinforcement learning (RL) due to the heterogeneity and dynamic importance of different modalities. Specifically, we observe that these issues make conventional RL methods difficult to learn a useful state representation in the end-to-end training with multimodal information. To address this, we propose a novel multimodal RL approach that can do multimodal alignment and importance enhancement according to their similarity and importance in terms of RL tasks respectively. By doing so, we are able to learn an effective state representation and consequentially improve the RL training process. We test our approach on several multimodal RL domains, showing that it outperforms state-of-the-art methods in terms of learning speed and policy quality.

AAMAS 2022

Multimodal Reinforcement Learning with Effective State Representation Learning

multimodal reinforcement learning.

deep reinforcement learning

reinforcement learning

**BRACIS** originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – **SBIA** (22 editions), and the Brazilian Symposium on Neural Networks – **SBRN** (13 editions). **ENIAC** is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. **STIL** is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: **CE-IA**, **CE-IC** and **CE-PLN**.

To access the 10th Brazilian Conference on Intelligent Systems (**BRACIS**), you need to register for the conference.  Please visit the Conference official site to register

[Register now](http://c4ai.inova.usp.br/bracis2021/)

Register now

BRACIS 2021

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – SBIA (22 editions), and the Brazilian Symposium on Neural Networks – SBRN (13 editions). 
ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. 
STIL is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: CE-IA, CE-IC and CE-PLN

poster

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).

A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).


In the Binary Networked Public Goods game, every player needs to decide if she participates in a public project whose utility is shared equally by the community. We study the problem of deciding if there exists a pure strategy Nash equilibrium (PSNE) in such games. The problem is already known to be NP-complete. We provide fine-grained analysis of this problem under the lens of parameterized complexity theory. We consider various natural graph parameters and show either W[1]-hardness or exhibit an FPT algorithm. Hence, our work allows us to understand the situations better where PSNE serves as a reliable solution concept versus where it is not.

On Parameterized Complexity of Binary Networked Public Goods Game


Augmented reality (AR) technologies have been applied to human-robot collaboration (HRC) domains to enable people to visualize the state of the robots.
Current AR-based visualization strategies are manually designed. This design process requires a lot of human efforts, and domain knowledge. When too little information is visualized, human users find the AR interface not useful; when too much is visualized, they find it difficult to process the visualized information. In this paper, we develop an intelligent AR agent that learns visualization policies (what to visualize, when, and how) from demonstrations. We developed a Unity-based platform for simulating warehouse environments where human-robot teammates work on collaborative delivery tasks. We have collected a dataset that includes 6000 demonstrations of visualizing robots' current and planned behaviors. Our results from experiments with real human participants show that, compared with competitive baselines from the literature, our learned visualization strategy significantly increases the efficiency of human-robot teams in delivery tasks.

Learning Visualization Policies of Augmented Reality for Human-Robot Collaboration

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1744.pdf

**Abstract:**



Autonomous Flight Arcade Challenge: Single- and Multi-Agent Learning Environments for Aerial Vehicles


When multiple humans and robots are moving in spaces like restaurants, hospitals, or banks, making the robot's movements easy to predict can help the humans co-navigate the space with the robots. Since people would be busy with their own goals, they are not paying close attention to the prior movements, or goals of multiple robots. So predictability from the robot's current position alone would help. With this in mind, we propose using an algorithm to lay out fixed paths for the different tasks the robots would do, such that predictability from only the current position alone is optimized, and motion costs are kept within acceptable bounds.

Minimizing Robot Navigation Graph For Position-Based Predictability By Humans


In network science, centrality indices are used to determine important nodes by virtue of their position in a network.
In social choice theory, voting rules are used to aggregate preferences of voters to determine the winners of an election.
Exploiting parallels between these two fields, we propose a novel approach to define network centrality indices based on voting rules.
Since formal properties of voting rules have been studied in much greater depth, this will not only lead to new applications of social choice theory but also facilitate a deeper understanding of centrality in networks.

Voting for Centrality


We study the problem of finding fair and efficient allocations of a set of indivisible items to a set of agents, where each item may be a good (positively valued) for some agents and a bad (negatively valued) for others, i.e., a mixed manna. As fairness notions, we consider arguably the strongest possible relaxations of envy-freeness and proportionality, namely envy-free up to any item (EFX and EFX_0), and proportional up to the maximin good or any bad (PropMX and PropMX_0). Our efficiency notion is Pareto-optimality (PO).

We study two types of instances: (i) Separable, where the item set can be partitioned into goods and bads, and (ii) Restricted mixed goods (RMG), where for each item j, every agent has either a non-positive value for j, or values j at the same v_j > 0. We obtain polynomial-time algorithms for the following:

* Separable instances: PropMX_0 allocation.
* RMG instances: Let pure bads be the set of items that everyone values negatively.
- PropMX allocation for general pure bads
- EFX+PropMX allocation for identically-ordered pure bads
- EFX+PropMX+PO allocation for identical pure bads.

Finally, if the RMG instances are further restricted to binary mixed goods where all the v_j's are the same, we strengthen the results to guarantee EFX_0 and PropMX_0 respectively.

(Almost) Envy-Free, Proportional and Efficient Allocations of an Indivisible Mixed Manna


Moving target defense (MTD) has emerged as a key technique that can be used in various security applications to reduce the threat of attackers by taking away their ability to perform reconnaissance and exploit vulnerabilities. However, most of the existing research in the field assumes unrealistic access to information about the attacker's motivations and/or actions when developing MTD strategies. Many of the existing approaches also assume complete knowledge regarding the vulnerabilities of a particular application and how each of these vulnerabilities can be exploited by an attacker. In this work, we propose an algorithm that generates effective MTD strategies for web applications that does not rely on prior knowledge about the attackers. Our approach assumes that the only information the defender receives about its own reward function, is via interaction with the attacker in a repeated game setting. We evaluate our algorithm using data which is mined from the National Vulnerability Database to show that it matches the performance of the state of the art techniques, despite using much less information.

Moving Target Defense under Uncertainty for Web Applications


For cooperative mutliagent reinforcement learning tasks, we propose a novel value factorization framework in the popular centralized training with decentralized execution paradigm, called multiagent Q-learning with sub-team coordination (QSCAN). This framework could flexibly exploit local coordination within sub-teams for effective factorization while honoring the individual-global-max (IGM) condition.
QSCAN encompasses the full spectrum of sub-team coordination according to sub-team size, ranging from the monotonic value function class to the entire IGM function class, with familiar methods such as QMIX and QPLEX located at the respective extremes of the spectrum. Empirical results show that QSCAN's performance dominates state-of-the-art methods in predator-prey tasks and the Switch challenge in MA-Gym.

Multiagent Q-learning with Sub-Team Coordination


This extended abstract presents an agent-based model to enable simulations to be performed of evacuations from a multi-floor building, thus including the movement in staircases. In the model, agents select exits and staircases based on familiarity level with the building layout, spatial distance to exits, preference to evacuate via the
main entrance/exit, and visibility of exits and staircases. The agentbased model has been validated using the empirical findings of an evacuation drill in the staircase of a four-floor building. The novel contribution of this research is the development of a more realistic agent-based model aimed at providing decision support for the layout design of buildings and evacuation behavioral management in emergency evacuation. That is, the agent-based model mimics people’s natural movement in staircases, and involves agents’ exit and staircase selection, and movement on floors. Simulations have been performed to investigate the effect on evacuation time of (i)
number of agents and their familiarity with the building layout, and (ii) exit selection of agents.

An Agent-based Model for Emergency Evacuation from a Multi-floor Building

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1765.pdf

**Abstract:**



Multimodal Reinforcement Learning with Effective State Representation Learning

Downloads

Next from AAMAS 2022

On Parameterized Complexity of Binary Networked Public Goods Game

Similar lecture

Zeroth Order Policy Search Methods for Global Optimization Problems: An Experimental Study

Multimodal Reinforcement Learning with Effective State Representation Learning

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

On Parameterized Complexity of Binary Networked Public Goods Game

Similar lecture

Zeroth Order Policy Search Methods for Global Optimization Problems: An Experimental Study

Downloads