United Kingdom

For intelligent conversational agents to speak about albums of images with users as humans do, they must be able to make sense of images as humans do. Computer vision methods can report directly observable information, but human beings care about more than the directly observable; they value holistic narratives that include affective and motivational evaluations, casual connections, and other inferred relationships from external knowledge. Drawing from theories in cognitive sensemaking and narrative coherence, we propose an approach for image sequence understanding that strives to generate and evaluate hypotheses about the relationships between people, events, and objects in images using commonsense knowledge, which are formed into a consistent network of hypotheses and observed facts via multi-objective optimization. The result is an enriched knowledge representation in the form of a knowledge graph which may later be used by a conversational agent.

AAMAS 2021

Image Sequence Understanding through Narrative Sensemaking


IWSSIP is an international conference that brings together researchers and developers from both academia and industry reporting the latest scientific and theoretical advances to discuss and debate major issues and state-of-the-art aspects on systems, signals and image processing.

###### For the session discussion, each author will be seated at individual breakout tables.  Please find which table they will be at here:
###### [QA table assignments in Gather.town](https://drive.google.com/file/d/1hBHOD_KF7NSsaTcpYcHCZslXJvQVxJQe/view?usp=sharing)

IWSSIP 2021

poster

AAMAS is the largest and most influential conference in the area of agents and multiagent systems, bringing together researchers and practitioners in all areas of agent technology and providing and internationally renowned high-profile forum for publishing and finding out about the latest developments in the field. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).  

![](https://assets.underline.io/uploads/markdown_image/1/image/ce4604bd54468a006164d16e080204fb.jpg)

Please click on the button below to register for this event.

If you are registered and see this page please check the account email you are logged in with. It must be the same as for the registration.

If you still have trouble accessing the contact please contact support at aamas2021@underline.io 

This paper describes a demonstration setup that integrates cognitive agents with the latest W3C standardization efforts for the Web of Things (WoT). The conceptual foundations of the implemented system are the integration of cognitive agent abstractions with W3C Web Things, which are generic abstractions of devices and virtual services that provide agents with various interaction affordances (e.g., actions, events). Together with the W3C WoT Scripting API, which is an ECMAScript-compatible API for W3C WoT environments, these standards allow JavaScript-based agents to be deployed and to operate in heterogeneous WoT environments. The agents can then be effectively distributed across the physical-virtual space in a write once, run anywhere manner: we deploy agents across a heterogeneous information system landscape that includes Web servers, browser-based front-ends, and constrained devices (microcontrollers). The deployment only requires minor platform-specific adjustments to consider resource and performance limitations on constrained devices. As a running example, we demonstrate a semi-autonomous assembly scenario with human-in-the-loop support.

Autonomous Agents on the Edge of Things

Autonomous agents that can engage in social interactions with a human is the ultimate goal of a myriad of applications. A key challenge in the design of these applications is to define the social behavior of the agent, which requires extensive content creation. In this research, we explore how we can leverage current state-of-the-art tools to make inferences about the emotional state of a character in a story as events unfold, in a coherent way. We propose a character role-labelling approach to emotion tracking that accounts for the semantics of emotions. We show that, by identifying actors and objects of events and considering the emotional state of the characters, we can achieve better performance in this task when compared to end-to-end approaches.

CHARET: Character-centered Approach to Emotion Tracking in Stories

We study perception in the scenario of an embodied agent equipped with first-person sensors and a continuous motor space with multiple degrees of freedom. We consider the commutative properties of action sequences with respect to sensory information perceived by such an embodied agent. We introduce the Sensory Commutativity Probability (SCP) criterion which measures how much an agent's degree of freedom affects the environment in embodied scenarios. We show how to compute this criterion in different environments, including realistic robotic setups and discuss how SCP and the commutative properties of action sequences can be used to learn about objects in the environment and improve sample-efficiency in Reinforcement Learning.

On the Sensory Commutativity of Action Sequences for Embodied Agents

Negotiation is one of the crucial processes for resolving conflicts between parties. In automated negotiation, agent designers mostly take opponent's offers and the remaining time into account while designing their strategies. While designing a negotiating agent interacting with a human directly, other information such as opponent's emotional changes during the negotiation can establish a better interaction and reach an admissible settlement for joint interests. Accordingly, this paper proposes a bidding strategy for humanoid robots, which incorporates their opponents' emotional states and awareness of the agent's changing behavior.

Solver Agent: Towards Emotional and Opponent-Aware Agent for Human-Robot Negotiation

We study fair and economically efficient allocation of indivisible goods among agents whose valuations are rank functions of matroids. Such valuations constitute a well-studied class of submodular functions (i.e., they exhibit a diminishing returns property) and model preferences in several resource-allocation settings. We prove that, for matroid-rank valuations, a social welfare-maximizing allocation that gives each agent her maximin share always exists. Furthermore, such an allocation can be computed in polynomial time. We establish similar existential and algorithmic results for the pairwise maximin share guarantee as well.

Existence and Computation of Maximin Fair Allocations Under Matroid-Rank Valuations

The task in judgment aggregation is to find a collective judgment set based on the views of individual judges about a given set of propositional formulas. One way of guaranteeing consistent outcomes is the use of sequential rules. In each round, the decision on a single formula is made either because the outcome is entailed by the already obtained judgment set, or, if this is not the case, by some underlying rule, e.g. the majority rule. Such rules are especially useful for cases, where the agenda is not fixed in advance, and formulas are added one by one. This paper investigates the computational complexity of winner determination under a family of sequential rules, and the manipulative influence of the processing order on the final outcome.

Complexity of Sequential Rules in Judgment Aggregation

We investigate a problem that lies at the intersection of three research areas, namely Automated Negotiation, Vehicle Routing, and Multi-Objective Optimization. Specifically, we investigate the scenario that multiple competing logistics companies aim to cooperate by delivering truck loads for one another, in order to improve efficiency and reduce the distance they drive. In order to do so, these companies need to find ways to exchange their truck loads such that each of them individually benefits. We present a new heuristic algorithm that, given one set of orders to deliver for each company, tries to find the set of all order-exchanges that are Pareto-optimal and individually rational. Furthermore, we present experiments based on real-world test data from two major logistics companies, which show that our algorithm is able to find hundreds of solutions in a matter of minutes.

A Heuristic Algorithm for Multi-Agent Vehicle Routing with Automated Negotiation

In this paper we present Probabilistic Control Argumentation Frameworks (PCAFs) that extend classical Control Argumentation Frameworks (CAFs) to take into account probabilistic information in the reasoning process. We show that probabilities can be used to optimally control CAFs that cannot be controlled otherwise. We introduce the notion of controlling power, that represents the probability that a control configuration reaches its target. A computational method based on Monte Carlo simulations for computing the controlling power of control configurations is defined. We experimentally show that PCAFs outperform w.r.t runtime classical CAFs and in a large number of situations they can reach the target with a high probability while the classical CAFs fail.

Probabilistic Control Argumentation Frameworks

We study the knapsack problem with group fairness constraints. The input of the problem consists of a knapsack of bounded capacity and a set of items. Each item belongs to a particular category and has an associated weight and value. The goal of this problem is to select a subset of items such that all categories are fairly represented, the total weight of the selected items does not exceed the capacity of the knapsack, and the total value is maximized. We study the fairness parameters such as the bounds on the total value of items from each category, the total weight of items from each category, and the total number of items from each category. We give approximation algorithms for these problems. We also give experimental validation for the efficiency of our algorithms. These fairness notions could also be extended to the min-knapsack problem.  The fair knapsack problems encompass various important problems, such as participatory budgeting, fair budget allocation, and advertising.

Group Fairness for Knapsack Problems

Artificial agents trained by deep reinforcement learning will likely encounter novel situations after deployment that were never seen during training. Our agent must be 
 to handle such situations well. However, if we cannot rely on the average training or validation reward as a metric, then how can we effectively evaluate robustness? We take inspiration from the practice of 
 in software engineering. Specifically, we suggest that when designing AI agents that collaborate with humans, designers should search for potential edge cases in 
 and 
 and write tests which check that the behavior of the agent in these edge cases is reasonable. We apply this methodology to build a suite of unit tests for the Overcooked-AI environment, and use this test suite to evaluate three proposals for improving robustness. We find that the test suite provides significant insight into the effects of these proposals that were generally not revealed by looking solely at the average validation reward. For our full paper, see https://arxiv.org/abs/2101.05507 arxiv.org/abs/2101.05507

Image Sequence Understanding through Narrative Sensemaking

Next from AAMAS 2021

Autonomous Agents on the Edge of Things

Similar lecture

Segmentation of signiﬁcant regions in retinal images: perspective of U-Net network through a comparative approach