Lavinia Gabriela Carpen, Tomy Acsente, Maria Adriana Acasandrei, Elena Matei, Claudia Gabriela Chilom, Diana Iulia Savu and Gheorghe Dinescu (September 27th 2019). The Interaction of Tungsten Dust with Human Skin Cells, Nanomaterials - Toxicity, Human Health and Environment, Simona Clichici, Adriana Filip and Gustavo M. do Nascimento, IntechOpen, DOI: 10.5772/intechopen.86632. 

Roger Parker (June 7th 2019). Human Behavior Modeling: The Necessity of Narrative, Computer Architecture in Industrial, Biomechanical and Biomedical Engineering, Lulu Wang and Liandong Yu, IntechOpen, DOI: 10.5772/intechopen.86686.

Masakazu Sato, Kaori Koga, Tomoyuki Fujii and Yutaka Osuga (June 27th 2018). Can Reinforcement Learning Be Applied to Surgery?, Artificial Intelligence - Emerging Trends and Applications, Marco Antonio Aceves-Fernandez, IntechOpen, DOI: 10.5772/intechopen.76146. 

New Zealand

&quot;Autonomous agents promise users of a personalized future, allowing
them to direct their attention to tasks most meaningful to them.
However, the demands of personalization stand unfulfilled by current
agent training paradigms such as machine learning, which
require many orders of data to train agents on a single task. In
sequential decision making domains, Reinforcement Learning (RL)
enables this need, when a priori training of desired behaviors is
intractable. Prior work has leveraged user input to train agents
by mapping them to numerical reward signals. However, recent
approaches have identified inconsistent human feedback as a bottleneck
to achieving best-case performance. In this work, we present
empirical evidence to show that human perception affected by
contrast effects distorts their feedback to Reinforcement Learning
agents. Through a set of studies involving 900 participants from
Amazon Mechanical Turk who were asked to give feedback to RL
agents, we show that participants significantly underrate an agent’s
actions after being exposed to an agent of higher competence on the
same task. To understand the significance of this effect on agent performance
during training, we then simulate trainers that underrate
actions of an agent based on past performance – creating a systematically
skewed feedback signal – integrated into an actor-critic
framework. Our results show that agent performance is reduced by
up to 98% in the presence of systematic skews in human feedback in
Atari environments. Our work provides a conceptual understanding
of a source of inconsistency in human feedback, thus informing the
design of human-agent interactions.&quot;

AAMAS 2020

Yesterday’s Reward is Today’s Punishment: Contrast Effects in Human Feedback to Reinforcement Learning Agents

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

"Autonomous agents promise users of a personalized future, allowing
them to direct their attention to tasks most meaningful to them.
However, the demands of personalization stand unfulfilled by current
agent training paradigms such as machine learning, which
require many orders of data to train agents on a single task. In
sequential decision making domains, Reinforcement Learning (RL)
enables this need, when a priori training of desired behaviors is
intractable. Prior work has leveraged user input to train agents
by mapping them to numerical reward signals. However, recent
approaches have identified inconsistent human feedback as a bottleneck
to achieving best-case performance. In this work, we present
empirical evidence to show that human perception affected by
contrast effects distorts their feedback to Reinforcement Learning
agents. Through a set of studies involving 900 participants from
Amazon Mechanical Turk who were asked to give feedback to RL
agents, we show that participants significantly underrate an agent’s
actions after being exposed to an agent of higher competence on the
same task. To understand the significance of this effect on agent performance
during training, we then simulate trainers that underrate
actions of an agent based on past performance – creating a systematically
skewed feedback signal – integrated into an actor-critic
framework. Our results show that agent performance is reduced by
up to 98% in the presence of systematic skews in human feedback in
Atari environments. Our work provides a conceptual understanding
of a source of inconsistency in human feedback, thus informing the
design of human-agent interactions."

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Explainability has been a core research topic in AI for decades and therefore it is surprising that the current concept of Explainable AI (XAI) seems to have been launched as late as 2016. This is a problem with current XAI research because it tends to ignore existing knowledge and wisdom gathered over decades or even centuries by other relevant domains. This paper presents the notion of Contextual Importance and Utility (CIU), which is based on known notions and methods of Decision Theory. CIU extends the notions of importance and utility for the non-linear models of AI systems and notably those produced by Machine Learning methods. CIU provides a universal and model-agnostic foundation for XAI.

Decision Theory meets Explainable AI

"This paper investigates the impact of changes in agents' beliefs coupled with dynamics in agents' meta-roles on the evolution of institutions. The study embeds agents' meta-roles in the BDI architecture. In this context, the study scrutinises the impact of cognitive dissonance in agents due to unfairness of institutions. To showcase our model, two
historical long-distance trading societies, namely Armenian merchants of New-Julfa and the English East India Company are simulated. Results show how change in roles of agents coupled with specific institutional characteristics leads to changes of the rules in the system."

Impact of meta-roles on the evolution of institutions

Towards Demystifying Subliminal Persuasivness: Using XAI-Techniques to Highlight Persuasive Markers of Public Speeches

Explaining the influence of prior knowlege on POMCP policies

The goal of multi-objective problems is to find solutions that balance different objectives. When solving multi-objective problems using reinforcement learning linear scalarisation techniques are generally used, however system expertise is required to optimise the weights for linear scalarisation. Thresholded Lexicographic Ordering (TLO) is one technique that avoids the need for an expert to specify weights; instead a system designer can directly specify a preferred ordering over objectives, along with a desired threshold value for each objective. In this presentation we propose a novel algorithm to dynamically set thresholds for use with TLO. We also present the first evaluation of TLO in a complex multi-objective multi-agent problem, the Dynamic Economic Emissions Dispatch domain. Our empirical results demonstrate that TLO with our dynamic thresholding algorithm achieves superior results when compared with a hand-tuned linear scalarisation method from previously published work.

Dynamic Thresholded Lexicographic Ordering

Online social networks are known to lack adequate multi-user privacy support. In this paper we present EXPRI, an agent architecture that aims to assist users in managing multi-user privacy conflicts. By considering the personal utility of sharing content and the individually preferred moral values of each user involved in the conflict, EXPRI identifies the best collaborative solution by applying practical reasoning techniques. Such techniques provide the agent with the cognitive process that is necessary for explainability. Furthermore, the knowledge gathered during the practical reasoning process allows EXPRI to engage in contrastive explanations.

Agent EXPRI: Licence to Explain

We propose a market-inspired bidding scheme for the assignment of paper reviews in large academic conferences. The primary contribution is an analysis of the incentives of reviewers during the bidding phases, when reviewers have private costs and some information about the demand for each paper, and want to obtain the best possible k papers for a predetermined k. We show that by assigning budgets to reviewers and a `price' for every paper that is (roughly) inversely proportional to its demand, the best response of a reviewer is to bid sincerely, i.e., on her most favorite papers, and match the budget even when it is not enforced. Finally, we show via extensive simulations on bidding data from real conferences, that our suggested bidding scheme would substantially improve the assignment, under several common assignment algorithms.

A Market-Inspired Bidding Scheme for Peer Review Paper Assignment

In the capacitated production planning problem, quantities of products need to be determined at consecutive periods within a given time horizon when product demands, costs, and production capacities vary through time. We focus on a general formulation of this problem where each product is produced in one step and setup cost is paid at each period of production. Additionally, products can be anticipated or backordered in respect to the demand period. We propose a computationally efficient decentralized approach based on the spillover effect relating to the accumulation of production costs of each product demand through time. The performance of the spillover algorithm is compared against the state-of-the-art mixed integer programming branch-and-bound solver CPLEX 12.8 considering optimality gap and computational time.

A Decentralized Multi-Agent Coordination Method for Dynamic and Constrained Production Planning

Reinforcement learning has been applied to numerous decision-making problems, but it requires careful specification of a reward function to represent the objective of the problem.
In many problems, it is difficult to represent the objective as a function, and thus it is easier to give expert demonstrations. Inverse reinforcement learning can be applied to such problems because it estimates the reward function from the expert's demonstrations.
Most existing inverse reinforcement learning methods assume that an expert gives demonstrations in a fixed environment, although the expert can provide demonstrations for a specific objective in multiple environments. For example, it is difficult to formulate a suitable reward function to represent driving a car, and the driver can give demonstrations in multiple situations. In such cases, normal practice is to use demonstrations in multiple environments to estimate the expert's reward. Herein, we formulate this problem based on a Bayesian inverse reinforcement learning framework and propose a mini-batch Markov chain Monte Carlo method. An advantage of our method is scalability. Our proposed method is scalable with respect to a number of environments in which expert demonstrations are generated. Experimental results show quantitatively that the proposed method outperforms existing inverse reinforcement learning methods.

Yesterday’s Reward is Today’s Punishment: Contrast Effects in Human Feedback to Reinforcement Learning Agents

Downloads

Next from AAMAS 2020

Decision Theory meets Explainable AI

Similar lecture

NQE: N-ary Query Embedding for Complex Query Answering over Hyper-relational Knowledge Graphs

Yesterday’s Reward is Today’s Punishment: Contrast Effects in Human Feedback to Reinforcement Learning Agents

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Decision Theory meets Explainable AI

Similar lecture

NQE: N-ary Query Embedding for Complex Query Answering over Hyper-relational Knowledge Graphs

Downloads