New Zealand


Representation learning on visualized input is an essential yet challenging task for deep reinforcement learning (RL). While training agents using deep reinforcement learning algorithms, we usually assume that the agent could extract appropriate features and take actions accordingly. However, learning essential and generalizable features based on high-dimensional input images is a great challenge. To help the RL agent learn more general and discriminative representation
among various states, we present cross-state self-constraint (CSSC). This novel technique regularizes representation learning by comparing state embedding similarities across different state-action pairs. We test our proposed method on the OpenAI Procgen benchmark with Rainbow and PPO and demonstrate significant improvement across most Procgen environments.

AAMAS 2022

Improving Generalization with Cross-State Behavior Matching in Deep Reinforcement Learning

behavioral matching

generalization

reinforcement learning

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

poster

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).

A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).


Measuring and comparing resilience is crucial for evaluating different algorithms’ performance. Existing resilience metrics focus on a system’s ability to maintain a particular state, but are inadequate to evaluate whether a system can achieve a novel state after an unexpected disturbance. The presented resilience power metric is used to analyze two best-of-N algorithms. Both algorithms exhibited high resilience power when changing the collective’s population size, but this result did not correlate with high overall task performance.

Measuring Resilience in Collective Robotic Algorithms


Covering option discovery has been developed to improve the exploration of reinforcement learning in single-agent scenarios with sparse reward signals, through connecting the most distant states in the embedding space provided by the Fiedler vector of the state transition graph. However, due to exponentially-large state space
resulted from multi-agent scenarios, existing researches on adopting options in multi-agent scenarios still rely on single-agent option discovery and fail to directly discover the joint options that can improve the connectivity of the joint state space of agents. In this paper, we show that it is indeed possible to directly compute
multi-agent options with collaborative exploratory behavior while still enjoying the ease of decomposition. Our key idea is to approximate the joint state space as a kronecker graph – the kronecker product of individual agents’ state transition graphs, based on which we can directly estimate the Fiedler vector of the joint
state space using the Laplacian spectrum of individual agents’ transition graphs. This decomposition enables us to efficiently construct multi-agent joint options by encouraging agents to connect the sub-goal joint states which are corresponding to the minimum or maximum values of the estimated joint Fiedler vector. The evaluation based on multi-agent collaborative tasks shows that the proposed
algorithm can successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options, in terms of both faster exploration and higher cumulative rewards.

Multi-agent Covering Option Discovery through Kronecker Product of Factor Graphs

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1592.pdf

**Abstract:**



Beyond Uninformed Search: Improving Branch-and-bound Based Acceleration Algorithms for Belief Propagation via Heuristic Strategies

We study mechanisms that allocate reviewers to papers in a fair and efficient manner. We model reviewer assignment as an instance of a fair allocation problem, presenting an extension of the classic round-robin mechanism, called Reviewer Round Robin (RRR). Round-robin mechanisms are a standard tool to ensure envy-free up to one item (EF1) allocations. However, fairness often comes at the cost of decreased efficiency. To overcome this challenge, we carefully select an approximately optimal round-robin order. Applying a relaxation of submodularity, $\gamma$-weak submodularity, we show that greedily inserting papers into an order yields a ${(1+\gamma^2)}$-approximation to the maximum welfare attainable by our round-robin mechanism under any order. Our approach outputs highly efficient EF1 allocations for three real conference datasets, offering comparable performance to state-of-the-art paper assignment methods in fairness, efficiency, and runtime, while providing the only EF1 guarantee.

I Will Have Order! Optimizing Orders for Fair Reviewer Assignment


Evacuation plans are designed to move people to safety in case of a disaster. It mainly consists of two components: routing and scheduling. However, the joint optimization of these two components with the goal of minimizing total evacuation time is a computationally hard problem, specifically when the problem instance is large. Moreover, often in disaster situations, there is uncertainty regarding the passability of roads throughout the evacuation time period. In this paper, we present a way to model the time-varying risk associated with roads in disaster situations. Then, we present a heuristic method that performs the joint optimization over routing and scheduling by iteratively solving subproblems of the same form. We use real world road network and population data from Harris County in Houston, Texas and apply our heuristic to find evacuation routes and schedules for the area. We show that the proposed method is able to find good solutions within a reasonable amount of time. We also perform agent-based simulations of the evacuation using these solutions to evaluate their quality and efficacy.

Data-driven Agent-based Models for Optimal Evacuation of Large Metropolitan Areas for Improved Disaster Planning


This study proposes a new scheme for implementing actual data into artificial market simulations at the level of trader agents. Because humans can introduce bias or overlook the important features of actual traders, we implemented the actual data and automated the strategy learning (imitating) of agents using machine learning (ML). We then ran artificial market simulations in the treader model, which imitates the actual trading behaviors in an ML architecture. Through this study, we demonstrate the potentials and limitations of the proposed scheme.

Implementation of Actual Data for Artificial Market Simulation


Individual rationality, which involves maximizing expected individual returns, does not always lead to high-utility individual or group outcomes in multi-agent problems. For instance, in multi-agent social dilemmas, Reinforcement Learning (RL) agents trained to maximize individual rewards converge to a low-utility mutually harmful equilibrium. In contrast, humans evolve useful strategies in such social dilemmas. Inspired by ideas from human psychology that attribute this behavior to the status-quo bias, we present a status-quo loss (𝑆𝑄𝐿𝑜𝑠𝑠) and the corresponding policy gradient algorithm that incorporates this bias in an RL agent. We demonstrate that agents trained with 𝑆𝑄𝐿𝑜𝑠𝑠 learn high-utility policies in several social dilemma matrix games (Prisoner’s Dilemma, Matching Pennies, Chicken Game). To apply SQLoss to visual input games where cooperation and defection are determined by a sequence of lower-level actions, we present GameDistill, an algorithm that reduces a visual input game to a matrix game. We empirically show how agents trained with SQLoss on GameDistill reduced versions of Coin Game and Stag Hunt learn high-utility policies. Finally, we show that 𝑆𝑄𝐿𝑜𝑠𝑠 extends to a 4-agent setting by demonstrating the emergence of cooperative behavior in the popular Braess’ paradox.

Status-quo policy gradient in Multi-Agent Reinforcement Learning

Artificial social agents (ASAs) are computer-based autonomous entities who interact with humans in a range of social roles, including advising, coaching, and consumer support in education and health. While there are many discussions around the ethical use of Artificial Intelligence in general, such as the AI4People Ethical Framework, the ethical ramifications that ASAs will have on human relationships as we share not only personal data but our inner most thoughts and feelings, is less explored. We conducted a study with 199 student participants exposed to Sam, an ASA which acts as a personal guide to support the student during their studies. During the interaction, Sam elicits private information, takes decisions for the user and speaks of its own study experiences. Our results indicate that (loss of) autonomy raised the strongest ethical concerns. These results confirm the importance of informed consent, transparency and accountability of ASAs and question the ethics
of false memories and emotion sharing.

The Ethical Acceptability of Artificial Social Agents


In robotics, data acquisition often plays a key part in unknown environment exploration. For example, storing information about the topography of the explored terrain can inform the decision-making process of the robots. Therefore, it is crucial to store these data safely and to make it available quickly to the operators of the robotic system. In a decentralized system like a swarm of robots, this entails several challenges. To address them, we propose RASS, a decentralized risk-aware swarm storage and routing mechanism, which relies exclusively on local information sharing between neighbours to establish storage and routing fitness. We test our system through thorough experiments in a physics-based simulator and obtain convincing reliability, routing speeds, and swarm storage capacity results.

RASS: Risk-Aware Swarm Storage

In multi-objective decision problems, the same state-action pair under different preference weights between the objectives, constitutes different optimal policies. The introduction of changing preference weights interferes with the convergence of the network, and can even stop the network from converging. In this work, we propose a novel experience sampling strategy for multi-objective RL problems, which samples transitions based on the weight and state similarities, to get the sampled experiences close to on-policy. We apply our sampling strategy in multi-objective deep RL algorithms on known benchmark problems, and show that this strongly improves performance.

Improving Generalization with Cross-State Behavior Matching in Deep Reinforcement Learning

Downloads

Next from AAMAS 2022

Measuring Resilience in Collective Robotic Algorithms

Similar lecture

Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions

Improving Generalization with Cross-State Behavior Matching in Deep Reinforcement Learning

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

Measuring Resilience in Collective Robotic Algorithms

Similar lecture

Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions

Downloads