United Kingdom

In this study, we leverage multi-objective reinforcement learning to create tunable agents, i.e., agents that can adopt a range of different behaviours according to the designer&#39;s preferences, without the need for retraining. We apply this technique to sequential social dilemmas, settings where there is inherent tension between individual and collective rationality. Learning a single fixed policy in such settings leaves one at a significant disadvantage if the opponents&#39; strategies change after learning is complete. In our work, we demonstrate empirically that the tunable agents framework allows easy adaption between cooperative and competitive behaviours in sequential social dilemmas without the need for retraining, allowing a single trained agent model to be adjusted to cater for a wide range of behaviours and opponent strategies.

AAMAS 2021

Tunable Behaviours in Sequential Social Dilemmas using Multi-Objective Reinforcement Learning

AAMAS is the largest and most influential conference in the area of agents and multiagent systems, bringing together researchers and practitioners in all areas of agent technology and providing and internationally renowned high-profile forum for publishing and finding out about the latest developments in the field. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).  

![](https://assets.underline.io/uploads/markdown_image/1/image/ce4604bd54468a006164d16e080204fb.jpg)

Please click on the button below to register for this event.

If you are registered and see this page please check the account email you are logged in with. It must be the same as for the registration.

If you still have trouble accessing the contact please contact support at aamas2021@underline.io 

In this study, we leverage multi-objective reinforcement learning to create tunable agents, i.e., agents that can adopt a range of different behaviours according to the designer's preferences, without the need for retraining. We apply this technique to sequential social dilemmas, settings where there is inherent tension between individual and collective rationality. Learning a single fixed policy in such settings leaves one at a significant disadvantage if the opponents' strategies change after learning is complete. In our work, we demonstrate empirically that the tunable agents framework allows easy adaption between cooperative and competitive behaviours in sequential social dilemmas without the need for retraining, allowing a single trained agent model to be adjusted to cater for a wide range of behaviours and opponent strategies.

poster

For intelligent conversational agents to speak about albums of images with users as humans do, they must be able to make sense of images as humans do. Computer vision methods can report directly observable information, but human beings care about more than the directly observable; they value holistic narratives that include affective and motivational evaluations, casual connections, and other inferred relationships from external knowledge. Drawing from theories in cognitive sensemaking and narrative coherence, we propose an approach for image sequence understanding that strives to generate and evaluate hypotheses about the relationships between people, events, and objects in images using commonsense knowledge, which are formed into a consistent network of hypotheses and observed facts via multi-objective optimization. The result is an enriched knowledge representation in the form of a knowledge graph which may later be used by a conversational agent.

Image Sequence Understanding through Narrative Sensemaking

Autonomous agents that can engage in social interactions with a human is the ultimate goal of a myriad of applications. A key challenge in the design of these applications is to define the social behavior of the agent, which requires extensive content creation. In this research, we explore how we can leverage current state-of-the-art tools to make inferences about the emotional state of a character in a story as events unfold, in a coherent way. We propose a character role-labelling approach to emotion tracking that accounts for the semantics of emotions. We show that, by identifying actors and objects of events and considering the emotional state of the characters, we can achieve better performance in this task when compared to end-to-end approaches.

CHARET: Character-centered Approach to Emotion Tracking in Stories

Negotiation is one of the crucial processes for resolving conflicts between parties. In automated negotiation, agent designers mostly take opponent's offers and the remaining time into account while designing their strategies. While designing a negotiating agent interacting with a human directly, other information such as opponent's emotional changes during the negotiation can establish a better interaction and reach an admissible settlement for joint interests. Accordingly, this paper proposes a bidding strategy for humanoid robots, which incorporates their opponents' emotional states and awareness of the agent's changing behavior.

Solver Agent: Towards Emotional and Opponent-Aware Agent for Human-Robot Negotiation

We study fair and economically efficient allocation of indivisible goods among agents whose valuations are rank functions of matroids. Such valuations constitute a well-studied class of submodular functions (i.e., they exhibit a diminishing returns property) and model preferences in several resource-allocation settings. We prove that, for matroid-rank valuations, a social welfare-maximizing allocation that gives each agent her maximin share always exists. Furthermore, such an allocation can be computed in polynomial time. We establish similar existential and algorithmic results for the pairwise maximin share guarantee as well.

Existence and Computation of Maximin Fair Allocations Under Matroid-Rank Valuations

The task in judgment aggregation is to find a collective judgment set based on the views of individual judges about a given set of propositional formulas. One way of guaranteeing consistent outcomes is the use of sequential rules. In each round, the decision on a single formula is made either because the outcome is entailed by the already obtained judgment set, or, if this is not the case, by some underlying rule, e.g. the majority rule. Such rules are especially useful for cases, where the agenda is not fixed in advance, and formulas are added one by one. This paper investigates the computational complexity of winner determination under a family of sequential rules, and the manipulative influence of the processing order on the final outcome.

Complexity of Sequential Rules in Judgment Aggregation

We study the problem of spending a budget that is common to n agents. Agents submit demands to a central planner who uses the budget to fund a subset of them. The utility of an agent is the part of the budget spent on her own accepted demands. In a fair solution, the successful demands of each agent would represent a 1/n fraction of the budget. However, this is rarely possible because every demand is indivisible, i.e. either accepted in its entirety or rejected. We are interested in worst-case bounds on the largest proportion of the budget that is dedicated to the least funded agent. Our approach is not to solve the corresponding max min problem for every instance, but to tackle the problem from a higher level. The size of the largest demand compared to the budget and the number of agents, are two parameters that significantly influence how much the worst-off agent gets. We propose worst-case bounds on the best utility of the least funded agent for the class of instances where the number of agents and the most expensive demand are fixed to given values. A characterization of this quantity is provided for 1 and 2 agents. For more than 2 agents, %our results we propose lower and upper bounds that constitute a 14/15-approximation of the optimal value. Every existence result is complemented with a polynomial algorithm that builds a feasible solution satisfying our bounds.

Worst-case Bounds for Spending a Common Budget

In this paper we present Probabilistic Control Argumentation Frameworks (PCAFs) that extend classical Control Argumentation Frameworks (CAFs) to take into account probabilistic information in the reasoning process. We show that probabilities can be used to optimally control CAFs that cannot be controlled otherwise. We introduce the notion of controlling power, that represents the probability that a control configuration reaches its target. A computational method based on Monte Carlo simulations for computing the controlling power of control configurations is defined. We experimentally show that PCAFs outperform w.r.t runtime classical CAFs and in a large number of situations they can reach the target with a high probability while the classical CAFs fail.

Probabilistic Control Argumentation Frameworks

We study the knapsack problem with group fairness constraints. The input of the problem consists of a knapsack of bounded capacity and a set of items. Each item belongs to a particular category and has an associated weight and value. The goal of this problem is to select a subset of items such that all categories are fairly represented, the total weight of the selected items does not exceed the capacity of the knapsack, and the total value is maximized. We study the fairness parameters such as the bounds on the total value of items from each category, the total weight of items from each category, and the total number of items from each category. We give approximation algorithms for these problems. We also give experimental validation for the efficiency of our algorithms. These fairness notions could also be extended to the min-knapsack problem.  The fair knapsack problems encompass various important problems, such as participatory budgeting, fair budget allocation, and advertising.

Group Fairness for Knapsack Problems

In recent years, machine learning (ML) models have been successfully applied in a variety of real-world applications. However, they are often complex and incomprehensible to human users. This can decrease trust in their outputs and render their usage in critical settings ethically problematic. As a result, several methods for explaining such ML models have been proposed recently, in particular for black-box models such as deep neural networks (NNs). Nevertheless, these methods predominantly explain model outputs in terms of inputs, disregarding the inner workings of the ML model computing those outputs. We present Argflow, a toolkit enabling the generation of a variety of ‘deep’ argumentative explanations (DAXs) for outputs of NNs on classification tasks.

Argflow: A Toolkit for Deep Argumentative Explanations for Neural Networks

Artificial agents trained by deep reinforcement learning will likely encounter novel situations after deployment that were never seen during training. Our agent must be 
 to handle such situations well. However, if we cannot rely on the average training or validation reward as a metric, then how can we effectively evaluate robustness? We take inspiration from the practice of 
 in software engineering. Specifically, we suggest that when designing AI agents that collaborate with humans, designers should search for potential edge cases in 
 and 
 and write tests which check that the behavior of the agent in these edge cases is reasonable. We apply this methodology to build a suite of unit tests for the Overcooked-AI environment, and use this test suite to evaluate three proposals for improving robustness. We find that the test suite provides significant insight into the effects of these proposals that were generally not revealed by looking solely at the average validation reward. For our full paper, see https://arxiv.org/abs/2101.05507 arxiv.org/abs/2101.05507

Next from AAMAS 2021

Image Sequence Understanding through Narrative Sensemaking

Similar lecture

Tunable Behaviours in Sequential Social Dilemmas using Multi-Objective Reinforcement Learning