Kunikazu Kobayashi, Koji Nakano, Takashi Kuremoto and Masanao Obayashi (February 1st 2010). Objective-based Reinforcement Learning System for Cooperative Behavior Acquisition, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8615. 

Stelmo Magalhaes Barros Netto, Vanessa Rodrigues Coelho Leite, Aristofanes Correa Silva, Anselmo Cardoso de Paiva and Areolino de Almeida Neto (January 1st 2008). Application on Reinforcement Learning for Diagnosis Based on Medical Image, Reinforcement Learning, Cornelius Weber, Mark Elshaw and Norbert Michael Mayer, IntechOpen, DOI: 10.5772/5291. 

Douglas S. McNair (February 9th 2018). Preventing Disparities: Bayesian and Frequentist Methods for Assessing Fairness in Machine-Learning Decision-Support Models, New Insights into Bayesian Inference, Mohammad Saber Fallah Nezhad, IntechOpen, DOI: 10.5772/intechopen.73176. 

New Zealand

Reinforcement learning has been applied to numerous decision-making problems, but it requires careful specification of a reward function to represent the objective of the problem.
In many problems, it is difficult to represent the objective as a function, and thus it is easier to give expert demonstrations. Inverse reinforcement learning can be applied to such problems because it estimates the reward function from the expert&#39;s demonstrations.
Most existing inverse reinforcement learning methods assume that an expert gives demonstrations in a fixed environment, although the expert can provide demonstrations for a specific objective in multiple environments. For example, it is difficult to formulate a suitable reward function to represent driving a car, and the driver can give demonstrations in multiple situations. In such cases, normal practice is to use demonstrations in multiple environments to estimate the expert&#39;s reward. Herein, we formulate this problem based on a Bayesian inverse reinforcement learning framework and propose a mini-batch Markov chain Monte Carlo method. An advantage of our method is scalability. Our proposed method is scalable with respect to a number of environments in which expert demonstrations are generated. Experimental results show quantitatively that the proposed method outperforms existing inverse reinforcement learning methods.

AAMAS 2020

Mini-batch Bayesian Inverse Reinforcement Learning for Multiple Dynamics

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

Reinforcement learning has been applied to numerous decision-making problems, but it requires careful specification of a reward function to represent the objective of the problem.
In many problems, it is difficult to represent the objective as a function, and thus it is easier to give expert demonstrations. Inverse reinforcement learning can be applied to such problems because it estimates the reward function from the expert's demonstrations.
Most existing inverse reinforcement learning methods assume that an expert gives demonstrations in a fixed environment, although the expert can provide demonstrations for a specific objective in multiple environments. For example, it is difficult to formulate a suitable reward function to represent driving a car, and the driver can give demonstrations in multiple situations. In such cases, normal practice is to use demonstrations in multiple environments to estimate the expert's reward. Herein, we formulate this problem based on a Bayesian inverse reinforcement learning framework and propose a mini-batch Markov chain Monte Carlo method. An advantage of our method is scalability. Our proposed method is scalable with respect to a number of environments in which expert demonstrations are generated. Experimental results show quantitatively that the proposed method outperforms existing inverse reinforcement learning methods.

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Levaraging Communication Topologies between Learning Agents in Deep Reinforcements Learning

Geographical clusters of undervaccinated populations have emerged in various parts of the United States in recent years. Public health response involves surveillance and field work, which is very resource-intensive. Given that public health resources are often limited, iden- tifying and rank-ordering critical clusters can help prioritize and allocate scarce resources for surveillance and quick intervention. We quantify the criticality of a cluster as the additional number of infections caused if the cluster is underimmunized. We focus on finding clusters that maximize this measure and develop efficient approximation algorithms for finding critical clusters by exploiting structural properties of the problem. Our methods involve solving a more general problem of maximizing a submodular function on a graph with connectivity constraints. We apply our methods to the state of Minnesota, where we find clusters with significantly higher criticality than those obtained by heuristics used in public health.

Finding Spatial Clusters Susceptible to Epidemic Outbreaks due to Undervaccination

Multi-agent Adversial Inverse Reinforcemetn Learning with Latent Variables

The Price of Anarchy of Self-Selection in Tullock Contests

Narrow the gap between distribution of task goals and virtual goals.

Hindsight Planner

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-processing biases -- an important component of human decision making which can help us better understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems, as well as various neuropsychiatric conditions associated with disruptions in normal reward processing. From the computational perspective, we observe that the proposed Split-QL model and its clinically inspired variants consistently outperform standard Q-Learning and SARSA methods, as well as recently proposed Double Q-Learning approaches, on simulated tasks with particular reward distributions, a real-world dataset capturing human decision-making in gambling tasks, and the Pac-Man game in a lifelong learning setting across different reward stationarities.

A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry

Policy gradient and actor-critic algorithms form the basis of many commonly used training techniques in deep reinforcement learning. Using these algorithms in multiagent environments poses problems such as nonstationarity and instability. In this paper, we first demonstrate that standard softmax-based policy gradient can be prone to poor performance in the presence of even the most benign nonstationarity. By contrast, it is known that the replicator dynamics, a well-studied model from evolutionary game theory, eliminates dominated strategies and exhibits convergence of the time-averaged trajectories to interior Nash equilibria in zero-sum games. Thus, using the replicator dynamics as a foundation, we derive an elegant one-line change to policy gradient methods that simply bypasses the gradient step through the softmax, yielding a new algorithm titled Neural Replicator Dynamics (NeuRD). NeuRD reduces to the exponential weights/Hedge algorithm in the single-state all-actions case. Additionally, NeuRD has formal equivalence to softmax counterfactual regret minimization, which guarantees convergence in the sequential tabular case. Importantly, our algorithm provides a straightforward way of extending the replicator dynamics to the function approximation setting. Empirical results show that NeuRD quickly adapts to nonstationarities, outperforming policy gradient significantly in both tabular and function approximation settings, when evaluated on the standard imperfect information benchmarks of Kuhn Poker, Leduc Poker, and Goofspiel.

Neural Replicator Dynamics: Multiagent Learning via Hedging Policy Gradients

The problem of finding optimal solutions of stochastic functions over continuous domains is common in several real-world applications, such as, e.g., advertisement allocation, dynamic pricing, and power control in wireless networks. The optimization process is customarily performed by selecting input points sequentially and receiving a noisy observation from the function. In this paper, we resort to the Multi-Armed Bandit approach, aiming at optimizing stochastic functions when keeping at a pace the regret (i.e., the loss incurred during the learning process). In particular, we focus on smooth stochastic functions, as it is known that any algorithm suffers from a constant per-round regret when the domain is continuous, and the function does not satisfy any kind of regularity. Our main original contribution is the provision of a general family of algorithms, which, under the mild assumption that stochastic functions are a realization of a Gaussian Process, provides a regret of the order of O(\sqrt{\gamma_T T}), being \gamma_T the maximum information gain and T the time horizon used for the learning process. Furthermore, we design a specific algorithm of our family, called DAGP-UCB, which exploits the structure of GPs to select the next arm to pull more effectively than the previous algorithms available in the state of the art, thus speeding up the learning process. In particular, we show the superior performance of DAGP-UCB in both synthetic and applicative settings, comparing it with the state-of-the-art algorithms.

Driving Exploration by Maximum Distribution in Gaussian Process Bandits

We investigate adversarial patrolling where the Defender is an autonomous device with a limited energy resource (e.g., a drone). Every eligible Defender's policy must prevent draining the energy resource before arriving to a refill station, and this constraint substantially complicates the problem of computing an efficient Defender's policy. Furthermore, the existing infinite-horizon models assume Attackers with unbounded patience willing to wait arbitrarily long for a good attack opportunity. We show this assumption is inappropriate in the setting with drones because here the expected waiting time for an optimal attack opportunity can be extremely large. To overcome this problem, we introduce and justify a new concept of impatient Attacker, and design a polynomial time algorithm for computing a Defender's policy achieving protection close to the optimal value against an impatient Attacker. Since our algorithm can quickly evaluate the protection achievable for various topologies of refill stations, we can also optimize their displacement.

Adversarial Patrolling with Drones

We tackle the problem of learning equilibria in simulation-based games. In such games, the players' utility functions cannot be described analytically, as they are given through a black-box simulator that can be queried to obtain noisy estimates of the utilities. This is the case in many real-world games in which a complete description of the elements involved is not available upfront, such as complex military settings and online auctions. In these situations, one usually needs to run costly simulation processes to get an accurate estimate of the game outcome. As a result, solving these games begets the challenge of designing learning algorithms that can find (approximate) equilibria with high confidence, using as few simulator queries as possible. Moreover, since running the simulator during the game is unfeasible, the algorithms must first perform a pure exploration learning phase and, then, use the (approximate) equilibrium learned this way to play the game. In this work, we focus on two-player zero-sum games with infinite strategy spaces. Drawing from the best arm identification literature, we design two algorithms with theoretical guarantees to learn maximin strategies in these games. The first one works in the fixed-confidence setting, guaranteeing the desired confidence level while minimizing the number of queries. Instead, the second algorithm fits the fixed-budget setting, maximizing the confidence without exceeding the given maximum number of queries. First, we formally prove delta-PAC theoretical guarantees for our algorithms under some regularity assumptions, which are encoded by letting the utility functions be drawn from a Gaussian process. Then, we experimentally evaluate our techniques on a testbed made of randomly generated games and instances representing simple real-world security settings.

Mini-batch Bayesian Inverse Reinforcement Learning for Multiple Dynamics

Downloads

Next from AAMAS 2020

Levaraging Communication Topologies between Learning Agents in Deep Reinforcements Learning

Similar lecture

CMVAE: Causal Meta VAE for Unsupervised Meta-Learning

Mini-batch Bayesian Inverse Reinforcement Learning for Multiple Dynamics

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Levaraging Communication Topologies between Learning Agents in Deep Reinforcements Learning

Similar lecture

CMVAE: Causal Meta VAE for Unsupervised Meta-Learning

Downloads