New Zealand


Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world. Multi-agent reinforcement learning (MARL) has the potential to achieve this goal, demonstrating success in a series of challenging problems. However, whilst these advances are significant, the vast majority of focus has been on the self-play paradigm. This often results in a coordination problem, caused by agents learning to make use of arbitrary conventions when playing with themselves. This means that even the strongest self-play agents may have very low cross-play with other agents, including other initializations of the same algorithm. In this paper we propose to solve this problem by adapting agent strategies on the fly, using a posterior belief over the other agents&#39; strategy. Concretely, we consider the problem of selecting a strategy from a finite set of previously trained agents, to play with an unknown partner. We propose an extension of the classic statistical technique, Gibbs sampling, to update beliefs about other agents and obtain close to optimal ad-hoc performance. Despite its simplicity, our method is able to achieve strong cross-play with unseen partners in the challenging card game of Hanabi, achieving successful ad-hoc coordination without knowledge of the partner&#39;s strategy a priori.


AAMAS 2022

On-the-fly Strategy Adaptation for ad-hoc Agent Coordination

multi-agent systems; cooperative multi-agent systems; multi-agent reinforcement learning; bayesian inference; gibbs sampling

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.


Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world. Multi-agent reinforcement learning (MARL) has the potential to achieve this goal, demonstrating success in a series of challenging problems. However, whilst these advances are significant, the vast majority of focus has been on the self-play paradigm. This often results in a coordination problem, caused by agents learning to make use of arbitrary conventions when playing with themselves. This means that even the strongest self-play agents may have very low cross-play with other agents, including other initializations of the same algorithm. In this paper we propose to solve this problem by adapting agent strategies on the fly, using a posterior belief over the other agents' strategy. Concretely, we consider the problem of selecting a strategy from a finite set of previously trained agents, to play with an unknown partner. We propose an extension of the classic statistical technique, Gibbs sampling, to update beliefs about other agents and obtain close to optimal ad-hoc performance. Despite its simplicity, our method is able to achieve strong cross-play with unseen partners in the challenging card game of Hanabi, achieving successful ad-hoc coordination without knowledge of the partner's strategy a priori.


poster

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).
<br>
<br>



A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).


Many real-world applications require an agent to make robust and deliberate decisions with multimodal information (e.g., robots with multi-sensory inputs). However, it is very challenging to train the agent via reinforcement learning (RL) due to the heterogeneity and dynamic importance of different modalities. Specifically, we observe that these issues make conventional RL methods difficult to learn a useful state representation in the end-to-end training with multimodal information. To address this, we propose a novel multimodal RL approach that can do multimodal alignment and importance enhancement according to their similarity and importance in terms of RL tasks respectively. By doing so, we are able to learn an effective state representation and consequentially improve the RL training process. We test our approach on several multimodal RL domains, showing that it outperforms state-of-the-art methods in terms of learning speed and policy quality.

Multimodal Reinforcement Learning with Effective State Representation Learning


Augmented reality (AR) technologies have been applied to human-robot collaboration (HRC) domains to enable people to visualize the state of the robots.
Current AR-based visualization strategies are manually designed. This design process requires a lot of human efforts, and domain knowledge. When too little information is visualized, human users find the AR interface not useful; when too much is visualized, they find it difficult to process the visualized information. In this paper, we develop an intelligent AR agent that learns visualization policies (what to visualize, when, and how) from demonstrations. We developed a Unity-based platform for simulating warehouse environments where human-robot teammates work on collaborative delivery tasks. We have collected a dataset that includes 6000 demonstrations of visualizing robots' current and planned behaviors. Our results from experiments with real human participants show that, compared with competitive baselines from the literature, our learned visualization strategy significantly increases the efficiency of human-robot teams in delivery tasks.

Learning Visualization Policies of Augmented Reality for Human-Robot Collaboration

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1744.pdf

**Abstract:**



Autonomous Flight Arcade Challenge: Single- and Multi-Agent Learning Environments for Aerial Vehicles


When multiple humans and robots are moving in spaces like restaurants, hospitals, or banks, making the robot's movements easy to predict can help the humans co-navigate the space with the robots. Since people would be busy with their own goals, they are not paying close attention to the prior movements, or goals of multiple robots. So predictability from the robot's current position alone would help. With this in mind, we propose using an algorithm to lay out fixed paths for the different tasks the robots would do, such that predictability from only the current position alone is optimized, and motion costs are kept within acceptable bounds.

Minimizing Robot Navigation Graph For Position-Based Predictability By Humans


In network science, centrality indices are used to determine important nodes by virtue of their position in a network.
In social choice theory, voting rules are used to aggregate preferences of voters to determine the winners of an election.
Exploiting parallels between these two fields, we propose a novel approach to define network centrality indices based on voting rules.
Since formal properties of voting rules have been studied in much greater depth, this will not only lead to new applications of social choice theory but also facilitate a deeper understanding of centrality in networks.

Voting for Centrality


We study the problem of finding fair and efficient allocations of a set of indivisible items to a set of agents, where each item may be a good (positively valued) for some agents and a bad (negatively valued) for others, i.e., a mixed manna. As fairness notions, we consider arguably the strongest possible relaxations of envy-freeness and proportionality, namely envy-free up to any item (EFX and EFX_0), and proportional up to the maximin good or any bad (PropMX and PropMX_0). Our efficiency notion is Pareto-optimality (PO).

We study two types of instances: (i) Separable, where the item set can be partitioned into goods and bads, and (ii) Restricted mixed goods (RMG), where for each item j, every agent has either a non-positive value for j, or values j at the same v_j > 0. We obtain polynomial-time algorithms for the following:

* Separable instances: PropMX_0 allocation.
* RMG instances: Let pure bads be the set of items that everyone values negatively.
- PropMX allocation for general pure bads
- EFX+PropMX allocation for identically-ordered pure bads
- EFX+PropMX+PO allocation for identical pure bads.

Finally, if the RMG instances are further restricted to binary mixed goods where all the v_j's are the same, we strengthen the results to guarantee EFX_0 and PropMX_0 respectively.

(Almost) Envy-Free, Proportional and Efficient Allocations of an Indivisible Mixed Manna


Moving target defense (MTD) has emerged as a key technique that can be used in various security applications to reduce the threat of attackers by taking away their ability to perform reconnaissance and exploit vulnerabilities. However, most of the existing research in the field assumes unrealistic access to information about the attacker's motivations and/or actions when developing MTD strategies. Many of the existing approaches also assume complete knowledge regarding the vulnerabilities of a particular application and how each of these vulnerabilities can be exploited by an attacker. In this work, we propose an algorithm that generates effective MTD strategies for web applications that does not rely on prior knowledge about the attackers. Our approach assumes that the only information the defender receives about its own reward function, is via interaction with the attacker in a repeated game setting. We evaluate our algorithm using data which is mined from the National Vulnerability Database to show that it matches the performance of the state of the art techniques, despite using much less information.

Moving Target Defense under Uncertainty for Web Applications


For cooperative mutliagent reinforcement learning tasks, we propose a novel value factorization framework in the popular centralized training with decentralized execution paradigm, called multiagent Q-learning with sub-team coordination (QSCAN). This framework could flexibly exploit local coordination within sub-teams for effective factorization while honoring the individual-global-max (IGM) condition.
QSCAN encompasses the full spectrum of sub-team coordination according to sub-team size, ranging from the monotonic value function class to the entire IGM function class, with familiar methods such as QMIX and QPLEX located at the respective extremes of the spectrum. Empirical results show that QSCAN's performance dominates state-of-the-art methods in predator-prey tasks and the Switch challenge in MA-Gym.

Multiagent Q-learning with Sub-Team Coordination


This extended abstract presents an agent-based model to enable simulations to be performed of evacuations from a multi-floor building, thus including the movement in staircases. In the model, agents select exits and staircases based on familiarity level with the building layout, spatial distance to exits, preference to evacuate via the
main entrance/exit, and visibility of exits and staircases. The agentbased model has been validated using the empirical findings of an evacuation drill in the staircase of a four-floor building. The novel contribution of this research is the development of a more realistic agent-based model aimed at providing decision support for the layout design of buildings and evacuation behavioral management in emergency evacuation. That is, the agent-based model mimics people’s natural movement in staircases, and involves agents’ exit and staircase selection, and movement on floors. Simulations have been performed to investigate the effect on evacuation time of (i)
number of agents and their familiarity with the building layout, and (ii) exit selection of agents.

An Agent-based Model for Emergency Evacuation from a Multi-floor Building

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1765.pdf

**Abstract:**



On-the-fly Strategy Adaptation for ad-hoc Agent Coordination

Downloads

Next from AAMAS 2022

Multimodal Reinforcement Learning with Effective State Representation Learning

Similar lecture

Multi-Document Summarization of Medical Studies

On-the-fly Strategy Adaptation for ad-hoc Agent Coordination

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

Multimodal Reinforcement Learning with Effective State Representation Learning

Similar lecture

Multi-Document Summarization of Medical Studies

Downloads