New Zealand


Wind farms suffer from the so-called wake effects: when some of the turbines are located in the wakes of other turbines, their power output is substantially reduced. These losses can be partially mitigated via actively changing the yaw from the individually optimal direction. Many of the existing wake control techniques have two major limitations: they use simplified wake models to optimize the control strategy, and they assume that the atmospheric conditions remain stable. In this paper, we address these limitations by applying reinforcement learning (RL) to the active wake control problem. RL forgoes the wake model entirely and learns the optimal control strategy based on the observed atmospheric conditions and the reward signal, in this case the power output of the farm. It also accounts for random transitions in the observations, such as turbulent fluctuations in the wind. To evaluate the benefits of RL for active wake control, we implement a simulator based on the state-of-the-art FLORIS model in the OpenAI gym format, making it readily available to the RL community. Next, we propose three different state-action representations of the active wake control problem and investigate their effect on the performance of RL-based wake control. Finally, we compare RL to a state-of-the-art wake control strategy based on FLORIS and show that RL is less sensitive to changes in unobservable data.

AAMAS 2022

Deep Reinforcement Learning for Active Wake Control

active wake control

wind energy

deep reinforcement learning

reinforcement learning

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).
<br>
<br>



A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

technical paper


In agent programming and engineering, learning has been explored so far mainly as an effective AI technique to tackle specific problems, improving agents' (and MAS as well) adaptability and autonomy at runtime. 

In this paper we consider a broader role of learning, impacting on the full spectrum of agent programming and engineering - from design to runtime. In particular, we introduce a perspective in which agent development is conceived as a kind developmental learning process of the agent itself. In that view, a software agent "is born" with some core domain-independent learning capabilities at the architectural level and is grown up in a proper learning environment to acquire the proper skills to be able then to achieve its designed objectives once it will be deployed, analogously to children in the human case.

The idea relies on - on the one hand - existing literature about how learning has been injected in cognitive architectures as developed in cognitive science (e.g. SOAR, LIDA), as well as in agent architecture such as BDI. On the other hand, the idea drawn inspiration from human science, in particular the literature investigating the role of learning in human development, children in particular, and the impact of capabilities on human autonomy and practical reasoning.

From a software engineering point of view, this idea is meant to be a conceptual baseline to explore general-purpose architectures and methodologies - eventually extending existing ones, such as the BDI - to seamlessly and systematically integrate both parts engineered by developers (e.g. plans) and parts learnt by the agent, eventually adopting different learning strategies.

Main challenges of this view include devising the model and features for the primitive domain-independent learning system to be injected in agent core architecture and reasoning cycle, to support both developmental/continual learning, and how to structure the learning process occurring in the learning environment during the design/development stage. In the paper, causal reasoning and learning are identified and discussed as a main reference to this purpose, to be integrated with practical reasoning as implemented by modern agent architectures and technologies.

"Go to the Children": Rethinking to Intelligent Agent Design and Programming in a Developmental Learning Perspective


In this paper we propose a novel approach to education and learning, in which each human being is associated with a digital agent that takes care of proactively searching for opportunities by monitoring job markets, and consulting requirements, educational options and challenges. Then, it adjusts the current educational program and coordinates the pathway facilitating the optimal desired outcome. We envision a Distributed Educational Agent Network (DEAN) of connected, trustworthy educational agents that securely communicate with each other to collectively carry out the above-mentioned tasks, discussing the benefits and current and future challenges associated with the implementation of the proposed agent-assisted educational approach.

Agent-Assisted Life-Long Education and Learning

We develop a game theoretic model of malware protection using the state-of-the-art sandbox method, to characterize and compute optimal defense strategies for anti-malware. We model the strategic interaction between developers of malware (M) and anti-malware (AM) as a two player game, where AM commits to a strategy of generating sandbox environments, and M responds by choosing to either attack or hide malicious activity based on the environment it senses. We characterize the condition for AM to protect all its machines, and propose two frameworks to solve the for optimal AM strategy in other cases. First, we provide quadratically constrained quadratic program (QCQP)-based optimization framework to solve for the optimal AM strategy. Second, we model the problem using generative adversarial networks where M uses a discriminator to detect its current environments, and AM uses a generator to generate sandboxes to defeat M's discriminator.

Anti-Malware Sandbox Games

We show that Game Description Language (GDL) can be used to describe some of the most commonly used test-beds in the automated negotiations literature, namely Genius and Colored Trails. This opens up an entirely new, declarative, approach to automated negotiation, in which a single algorithm can negotiate over a very broad class of different negotiation domains. We formally prove that the set of possible agreements of any negotiation domain from Genius (either linear or non-linear) can be modeled as a set of strategies over a deterministic extensive-form game that can be described efficiently in GDL. Furthermore, we show experimentally that, given only this GDL description, we can explore the agreement space efficiently using entirely generic domain-independent algorithms. In addition, we show that the same also holds for negotiation domains in the Colored Trails framework. This means we have the basic ingredients to implement a single negotiating agent that is capable of negotiating over many different kinds of negotiation domains, including Genius and Colored Trails.

GDL as a Unifying Domain Description Language for Declarative Automated Negotiation (Extended Abstract)

We propose and study a general framework for modelling and formal reasoning about multi-agent systems and, in particular, multi-stage games where both quantitative and qualitative objectives and constraints are involved. Our models enrich concurrent game models with payoffs and guards on actions associated with each state of the model. We propose a quantitative extension of the logic ATL* that enables the combination of quantitative and qualitative reasoning. We illustrate the framework with some examples and then consider the model-checking problems arising in our framework and establish some general undecidability and decidability results for them.

Combining quantitative and qualitative reasoning in concurrent multi-player games


Automated Negotiation is attracting more attention from researchers recently
as it is becoming more relevant to industrial and business applications with
increased reliance on automated systems.
Most research in this area assumes either a
single negotiation thread with a well-defined utility function for each
agent involved or a set of concurrent negotiations with an ordering of
outcomes in each local negotiation. In this paper, we consider an agent
engaged in a set of concurrent negotiations with a utility function
defined only for the complete set of agreements in all of them and
no locally defined ordering of outcomes in any negotiation independent from what
happens in the others. We argue that this problem setting is interesting both 
from the academic and the industrial points of view. The paper then
presents an algorithm that allows such agent to maximize its expected
global utility by orchestrating its behavior in all negotiation
threads. The performance of the proposed method is analyzed theoretically
and empirically using simulation in the context of a trading market.

Concurrent Negotiations with Global Utility Functions


We introduce a practical and scalable algorithm (PALMA) for solving one of the fundamental problems of multi-agent systems -- finding matches and allocations -- in unboundedly large settings (e.g., resource allocation in urban environments, mobility-on-demand systems, etc.), while providing strong worse-case privacy guarantees. PALMA is decentralized, runs on-device, requires no inter-agent communication, and converges in constant time under reasonable assumptions. We evaluate PALMA in a mobility-on-demand and a paper assignment scenario, using real data in both, and demonstrate that it provides a strong level of privacy (epsilon < 1 and median as low as epsilon = 0.5 across agents) and high quality matchings (up to 86% of the non-private optimal).

A Distributed Differentially Private Algorithm for Resource Allocation in Unboundedly Large Settings


Many real-world scenarios involve teams of agents who have to coordinate their policies to achieve a shared goal. Most previous studies mainly focus on learning decentralized control policies to maximize a common reward and rarely consider the coordinated control policies, which is critical in dynamic, complicated environments. Out of the fully decentralized rules, we propose a graph-based coordination strategy (GCS) which consists of a graph generator and a factored coordination policy. We adopt an encoder-decoder model as the generator to construct a directed acyclic graph (DAG), dynamically representing the underlying decision structure. We also apply the DAGness and depth constrained optimization in the graph generator to balance efficiency and performance. To exploit the underlying decision structure, we investigate the factored coordination policy to obtain the optimally coordinated behaviors among agents. We train the graph generator and the factored coordination policy simultaneously to maximize the discounted return. Empirical evaluations on Collaborative Gaussian Squeeze, Cooperative Navigation, and Google Research Football demonstrate the superiority of the proposed method.

GCS: Graph-Based Coordination Strategy for Multi-Agent Reinforcement Learning


High-performing teams learn intelligent and efficient communication and coordination strategies to maximize their joint utility. These teams implicitly understand the different roles of heterogeneous team members and adapt their communication protocols accordingly. Multi-Agent Reinforcement Learning (MARL) seeks to develop computational methods for synthesizing such coordination strategies, but formulating models for heterogeneous teams with different state, action, and observation spaces has remained an open problem. Without properly modeling agent heterogeneity, as in prior MARL work that leverages homogeneous graph networks, communication becomes less helpful and can even deteriorate the cooperativity and team performance. We propose Heterogeneous Policy Networks (HetNet) to learn efficient and diverse communication models for coordinating cooperative heterogeneous teams. Building on heterogeneous graph-attention networks, we show that HetNet not only facilitates learning heterogeneous collaborative policies per existing agent-class but also enables end-to-end training for learning highly efficient binarized messaging. Our empirical evaluation shows that HetNet sets a new state of the art in learning coordination and communication strategies for heterogeneous multi-agent teams by achieving an 8.1% to 98.7% performance improvement over the next-best baseline across multiple domains while simultaneously achieving a 200x reduction in the required communication bandwidth.

Learning Efficient Diverse Communication for Cooperative Heterogeneous Teaming


In online advertising, search engines sell ad placements for keywords continuously through auctions. This problem can be seen as an infinitely repeated game since the auction is executed whenever a user performs a query with the keyword. As advertisers may frequently change their bids, the game will have a large set of equilibria with potentially complex strategies. In this paper, we propose the use of natural strategies for reasoning in such setting as they are processable by artificial agents with limited memory and/or computational power as well as understandable by humans users. To reach this goal, we introduce a quantitative version of Strategy Logic with natural strategies in the setting of imperfect information. In a first step, we show how to model strategies for repeated keyword auctions and take advantage of the model for proving properties evaluating this game. In a second step, we study the logic in relation to the distinguishable power, expressivity, and model-checking complexity for strategies with and without recall.

Deep Reinforcement Learning for Active Wake Control

Downloads

Next from AAMAS 2022

"Go to the Children": Rethinking to Intelligent Agent Design and Programming in a Developmental Learning Perspective

Similar lecture

Multimodal Reinforcement Learning with Effective State Representation Learning

Deep Reinforcement Learning for Active Wake Control

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

"Go to the Children": Rethinking to Intelligent Agent Design and Programming in a Developmental Learning Perspective

Similar lecture

Multimodal Reinforcement Learning with Effective State Representation Learning

Downloads