New Zealand


PointGoal navigation has seen significant recent interest and progress,
spurred on by the Habitat platform and associated challenge.
In this paper, we study PointGoal navigation under both a sample
budget (75 million frames) and a compute budget (1 GPU for 1 day).
We conduct an extensive set of experiments, cumulatively totaling
over 50,000 GPU-hours, that let us identify and discuss a number of
ostensibly minor but significant design choices – the advantage esti-
mation procedure (a key component in training), and visual encoder
architecture. Overall, these design choices lead to considerable and
consistent improvements. Under a sample budget, performance for
RGB-D agents improves 3 SPL on Gibson (4% relative improvement)
and 20 SPL on Matterport3D (43% relative improvement). Under a
compute budget, performance for RGB-D agents improves by 3 SPL
on Gibson (5% relative improvement) and 15 SPL on Matterport3D
(50% relative improvement). Our findings and recommendations
will serve to make the community’s experiments more efficient – to
reach 50 SPL with RGB-D on Matterport3D, they reduce the samples
needed by 3x and the training time 2x.

AAMAS 2022

How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget

navigation

embodied ai

reinforcement learning

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

ACL 2022

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

poster

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).

A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

Multiagent self-assembly allows collectives to reach areas otherwise inaccessible to any particular agent. However, the coordination of this collective is not trivial, so each agent’s position in the structure is usually determined apriori. In our approach, we take inspiration from army ants and use a simulated model of the Eciton Robotica robot to form emergent structures with bio-inspired local rules. We demonstrate that by coupling this with traffic control, we can induce the formation of a structure and control certain characteristics without pre-computed paths or central coordination.

Influencing Emergent Self-Assembled Structures in Robotic Collectives Through Traffic Control

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1536.pdf

**Abstract:**



Can Algorithms be Explained Without Compromising Efficiency? The Benefits of Detection and Imitation in Strategic Classification


Representation learning on visualized input is an essential yet challenging task for deep reinforcement learning (RL). While training agents using deep reinforcement learning algorithms, we usually assume that the agent could extract appropriate features and take actions accordingly. However, learning essential and generalizable features based on high-dimensional input images is a great challenge. To help the RL agent learn more general and discriminative representation
among various states, we present cross-state self-constraint (CSSC). This novel technique regularizes representation learning by comparing state embedding similarities across different state-action pairs. We test our proposed method on the OpenAI Procgen benchmark with Rainbow and PPO and demonstrate significant improvement across most Procgen environments.

Improving Generalization with Cross-State Behavior Matching in Deep Reinforcement Learning


Measuring and comparing resilience is crucial for evaluating different algorithms’ performance. Existing resilience metrics focus on a system’s ability to maintain a particular state, but are inadequate to evaluate whether a system can achieve a novel state after an unexpected disturbance. The presented resilience power metric is used to analyze two best-of-N algorithms. Both algorithms exhibited high resilience power when changing the collective’s population size, but this result did not correlate with high overall task performance.

Measuring Resilience in Collective Robotic Algorithms


Covering option discovery has been developed to improve the exploration of reinforcement learning in single-agent scenarios with sparse reward signals, through connecting the most distant states in the embedding space provided by the Fiedler vector of the state transition graph. However, due to exponentially-large state space
resulted from multi-agent scenarios, existing researches on adopting options in multi-agent scenarios still rely on single-agent option discovery and fail to directly discover the joint options that can improve the connectivity of the joint state space of agents. In this paper, we show that it is indeed possible to directly compute
multi-agent options with collaborative exploratory behavior while still enjoying the ease of decomposition. Our key idea is to approximate the joint state space as a kronecker graph – the kronecker product of individual agents’ state transition graphs, based on which we can directly estimate the Fiedler vector of the joint
state space using the Laplacian spectrum of individual agents’ transition graphs. This decomposition enables us to efficiently construct multi-agent joint options by encouraging agents to connect the sub-goal joint states which are corresponding to the minimum or maximum values of the estimated joint Fiedler vector. The evaluation based on multi-agent collaborative tasks shows that the proposed
algorithm can successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options, in terms of both faster exploration and higher cumulative rewards.

Multi-agent Covering Option Discovery through Kronecker Product of Factor Graphs

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1592.pdf

**Abstract:**



Beyond Uninformed Search: Improving Branch-and-bound Based Acceleration Algorithms for Belief Propagation via Heuristic Strategies

We study mechanisms that allocate reviewers to papers in a fair and efficient manner. We model reviewer assignment as an instance of a fair allocation problem, presenting an extension of the classic round-robin mechanism, called Reviewer Round Robin (RRR). Round-robin mechanisms are a standard tool to ensure envy-free up to one item (EF1) allocations. However, fairness often comes at the cost of decreased efficiency. To overcome this challenge, we carefully select an approximately optimal round-robin order. Applying a relaxation of submodularity, $\gamma$-weak submodularity, we show that greedily inserting papers into an order yields a ${(1+\gamma^2)}$-approximation to the maximum welfare attainable by our round-robin mechanism under any order. Our approach outputs highly efficient EF1 allocations for three real conference datasets, offering comparable performance to state-of-the-art paper assignment methods in fairness, efficiency, and runtime, while providing the only EF1 guarantee.

I Will Have Order! Optimizing Orders for Fair Reviewer Assignment


Evacuation plans are designed to move people to safety in case of a disaster. It mainly consists of two components: routing and scheduling. However, the joint optimization of these two components with the goal of minimizing total evacuation time is a computationally hard problem, specifically when the problem instance is large. Moreover, often in disaster situations, there is uncertainty regarding the passability of roads throughout the evacuation time period. In this paper, we present a way to model the time-varying risk associated with roads in disaster situations. Then, we present a heuristic method that performs the joint optimization over routing and scheduling by iteratively solving subproblems of the same form. We use real world road network and population data from Harris County in Houston, Texas and apply our heuristic to find evacuation routes and schedules for the area. We show that the proposed method is able to find good solutions within a reasonable amount of time. We also perform agent-based simulations of the evacuation using these solutions to evaluate their quality and efficacy.

Data-driven Agent-based Models for Optimal Evacuation of Large Metropolitan Areas for Improved Disaster Planning


This study proposes a new scheme for implementing actual data into artificial market simulations at the level of trader agents. Because humans can introduce bias or overlook the important features of actual traders, we implemented the actual data and automated the strategy learning (imitating) of agents using machine learning (ML). We then ran artificial market simulations in the treader model, which imitates the actual trading behaviors in an ML architecture. Through this study, we demonstrate the potentials and limitations of the proposed scheme.

Implementation of Actual Data for Artificial Market Simulation


Individual rationality, which involves maximizing expected individual returns, does not always lead to high-utility individual or group outcomes in multi-agent problems. For instance, in multi-agent social dilemmas, Reinforcement Learning (RL) agents trained to maximize individual rewards converge to a low-utility mutually harmful equilibrium. In contrast, humans evolve useful strategies in such social dilemmas. Inspired by ideas from human psychology that attribute this behavior to the status-quo bias, we present a status-quo loss (𝑆𝑄𝐿𝑜𝑠𝑠) and the corresponding policy gradient algorithm that incorporates this bias in an RL agent. We demonstrate that agents trained with 𝑆𝑄𝐿𝑜𝑠𝑠 learn high-utility policies in several social dilemma matrix games (Prisoner’s Dilemma, Matching Pennies, Chicken Game). To apply SQLoss to visual input games where cooperation and defection are determined by a sequence of lower-level actions, we present GameDistill, an algorithm that reduces a visual input game to a matrix game. We empirically show how agents trained with SQLoss on GameDistill reduced versions of Coin Game and Stag Hunt learn high-utility policies. Finally, we show that 𝑆𝑄𝐿𝑜𝑠𝑠 extends to a 4-agent setting by demonstrating the emergence of cooperative behavior in the popular Braess’ paradox.

How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget

Downloads

Next from AAMAS 2022

Influencing Emergent Self-Assembled Structures in Robotic Collectives Through Traffic Control

Similar lecture

VISITRON: Visual Semantics-Aligned Interactively Trained Object-Navigator

How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

Influencing Emergent Self-Assembled Structures in Robotic Collectives Through Traffic Control

Similar lecture

VISITRON: Visual Semantics-Aligned Interactively Trained Object-Navigator

Downloads