Spain

Reinforcement learning is an efficient, widely used machine learning technique that performs well when the state and action spaces are reasonable. This is rarely the case regarding control-related problems, as for instance controlling traffic signals. Here, the state space can be very large. In order to deal with the curse of dimensionality, a rough discretization of such space can be used but this is effective just up to a certain point. A way to mitigate this is to use techniques that generalize the state space such as function approximation. In this paper, a linear function approximation is used. Specifically, SARSA(λ) with  Fourier basis features is implemented to control traffic signals in the agent-based transport simulation MATSim. The results are compared not only to trivial controllers such as fixed-time, but also to state-of-the-art rule-based adaptive methods. It is concluded that SARSA(λ) with Fourier basis features is able to outperform such methods, especially in scenarios with varying traffic demands.

ECAI 2020

A reinforcement learning approach with Fourier basis linear function approximation for traffic signal control

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS 2020

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

technical paper

ECAI2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI, including:




**An extensive programme of workshops, tutorials, and invited speakers**

**The Prestigious Applications of Intelligent Systems conference (PAIS)**

**The Frontiers in AI track sessions**

**The Lab To Market event**

**The Women in AI time**

**The EU Challenges forum**

… and many more!

ECAI 2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI.

"* Université Lumière Lyon 2;
roua.elchamaa@univ-lyon2.fr;
yacine.ouzrout@univ-lyon2.fr;
† Saint-Joseph University (ESIB);
rima.kilany@usj.edu.lb;
;
‡ Univ. Lyon, University Claude;
Bernard Lyon 1;
France Email: baudouin.dafflon@univ-lyon1.fr;"

Semantic Traffic Data Analysis for a Local Leader Election Algorithm (LLEA)

Travel apps become more and more popular giving information about the current traffic state to drivers who then adapt their route choice. In commuting scenarios, where people repeatedly travel between a particular origin and destination, learning effects add to this information. In this paper, we analyse the effects on the overall network, if adaptive driver agents share their aggregated experience about route choice in a reinforcement learning (Q-learning) setup. Drivers share what they have learnt about the system, not just information about their current travel times. We can show in a standard scenario that experience sharing can improve convergence times for adaptive driver agents.

Experience Sharing in a Traffic Scenario

"On-Demand Transport (ODT) systems have attracted increasing attention in recent years. 
  Traditional centralized dispatching can achieve optimal solutions with NP-Hard complexity, making it unsuitable for online and dynamic problems. Decentralized approaches can achieve fast feasible solution at run-time with no guarantee on the quality. Starting from feasible not optimal solution, we present in this work a new decentralized, insertion heuristic algorithm to solve a special case of the dynamic Dial-A-Ride-Problem (DARP)  as an ODT system, in which vehicles communicate via Vehicle-to-vehicle communication (V2V)  and take decentralized decisions. In our solution model we add an optimization phase in order to improve the quality  of global solution achieved by decentralized decision at run-time by exchanging resources between vehicles. We evaluate and analyze the promising results of our contributed technique on synthetic data for taxis operating in Saint-Étienne city, against a classical greedy approach."

Decentralized Insertion Heuristic with Runtime Optimization for On-demand Transport Scheduling

"Nested Rollout Policy Adaptation (NRPA) is a Monte Carlo search
algorithm that learns a playout policy in order to solve a single
player game. In this paper we apply NRPA to the vehicle routing problem.
This problem is important for large companies that have to manage  
a fleet of vehicles on a daily basis. Real problems are often too large to be solved 
exactly. The algorithm is applied to standard problem of the literature 
and to the specific problems of EDF (Electricité De France, the main french electric utility company). These specific problems have peculiar constraints. 
NRPA gives better result than the algorithm previously used by EDF."

Monte Carlo Vehicle Routing

"When robots undertake tasks in adversarial environments in which they must cooperate with one another (e.g., military applications or the RoboCup Competition), they are at risk for being deceived by competitors. Competi-tors can misdirect the team to gain a positional advantage. Our lab is explor-ing ways in which teams of robots can be misdirected, in part, so counter-deception strategies can be devised. This paper explores how robot shills can be used to misdirect a multi-robot team. It defines behaviors for the agents to be deceived (the mark agents) using the multi-agent coordination literature as well as behaviors for the deceiving team (the shills and lead agent). These behaviors were implemented and simulations were run for a variety of conditions. The results show how shills can facilitate misdirec-tion in certain circumstances. They provide insights into enhancing multi-robot team deception.
"

Wolves in Sheep’s Clothing: Using Shill Agents to Misdirect Multi-Robot Teams

"In 2000, it was predicted that artificially intelligent agents would inevitably
become deceptive. Today, in a world seemingly awash with fake news
and in which we hand over control of our home environments to faux-human
smart devices, it is timely to review the types of deception that have actually
emerged. By reference to examples from diverse branches of AI, we classify research on deception into five novel categories, which we describe according to the human characteristic with which they most closely align: imitating, obfuscating,
tricking, calculating and reframing. We offer this as a way for those within AI
to recognise connections across the discipline and to suggest how AI-instigated
deceptions may be understood by those outside."

Characterising Deception in AI: a survey

Drawing on an ecological perspective, we contend that research into deception in AI needs to consider not only the cognitive structures of would-be deceptive agents but also the nature of the environments in which they act. To illustrate this approach, we report work-in-progress to design a game called MindTrails, played between a software agent and a human opponent, that is informed by the principles of stage magic to embed deceptive possibilities into its game world. MindTrails is intended to have well-defined elements and rules, while being complex enough to afford a rich range of deceptive behaviours. In this way, it allows us to more precisely articulate some of the deceptive principles of the stage magician and render them more accessible to AI methods and researchers.

The role of environments in affording deceptive behaviour: some preliminary insights from stage magic

A description about the topic, E-friend overview, theory of emotions, description of DAM module and an example of interaction between E-friend and student

E-friend: A logical-based AI Agent System Chat-bot for EmotionalWell-being and Mental Health (Paper #5)

"The goal of the work presented in this paper is to provide techniques to challenge the results of an algorithmic decision system relying on machine learning. We highlight the differences between explanations and justifications and outline a framework to generate evidence to support or to dismiss challenges. We also present the results of a preliminary study to assess users’ perception of the different types of challenges proposed here and their benefits to detect incorrect results.
"

Towards a framework for challenging ML-based decisions

Previous work in the social sciences, psychology and linguistics has show that liars have some control over the content of their stories, however their underlying state of mind may "leak out" through the way that they tell them. To the best of our knowledge, no previous systematic effort exists in order to describe and model deception language for Brazilian Portuguese. To fill this important gap, we carry out a pioneering empirical linguistic study on false statements in Brazilian news. We methodically analyze linguistic features using the Fake.Br corpus, which includes both fake and true news. The results show that they present substantial lexical, syntactic and semantic variations, as well as punctuation and emotion distinctions.

A reinforcement learning approach with Fourier basis linear function approximation for traffic signal control

Downloads

Next from ECAI 2020

Semantic Traffic Data Analysis for a Local Leader Election Algorithm (LLEA)

Similar lecture

Toll-Based Learning for Minimising Congestion under Heterogeneus Preferences

A reinforcement learning approach with Fourier basis linear function approximation for traffic signal control

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ECAI 2020

Semantic Traffic Data Analysis for a Local Leader Election Algorithm (LLEA)

Similar lecture

Toll-Based Learning for Minimising Congestion under Heterogeneus Preferences

Downloads