New Zealand

Dennis Barrios-Aranibar and Luiz M. G. Gon&#231;alves (January 1st 2009). Influence Value Q-Learning: A Reinforcement Learning Algorithm for Multi Agent Systems, Theory and Novel Applications of Machine Learning, Meng Joo Er and Yi Zhou, IntechOpen, DOI: 10.5772/6675.

Thomas Behan, Ziayi Liao and Lian Zhao (November 1st 2009). Integer Neural Networks On Embedded Systems, Recent Advances in Technologies, Maurizio A Strangio, IntechOpen, DOI: 10.5772/7426. 

We introduce a model for agent-environment systems where the agents are implemented via feed-forward ReLU neural networks and the environment is non-deterministic. We study the verification problem of such systems against CTL properties. We introduce a bounded fragment of CTL and show its usefulness in identifying shallow bugs in the system. We present a novel parallel algorithm for MILP-based verification of agent-environment systems, present an implementation, and report the experimental results obtained against a variant of the VerticalCAS use-case.


AAMAS 2020

Formal Verification of Neural Agents in Non-deterministic Environments

learning-enabled multi-agent systems

formal verification

neural networks

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

AAAI 2025

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Coupon has been a major marketing tool that promotes sales for new and repeated buyers and proven effective in numerous realistic scenarios. In this paper, we mainly focus on the problem of designing coupons to maximize the revenue in second-price auction. 

Firstly, we derive the dominant strategies of bidders if they are provided with coupons in second-price auction and prove that the revenue optimization problem with coupons for all the bidders is NP-complete. Secondly, we cast the problem of designing coupons to maximize revenue into a learning framework. With well-designed loss functions, we perform theoretical analysis of its properties and propose corresponding algorithms to solve the problem. Finally, with both synthetic data and industrial data, extensive experiments are conducted to demonstrate their effectiveness.


Learning to Design Coupons in Online Advertising Markets

Advertising has become a dominant source of revenue generation on the Internet. Billions of advertisement slots are sold via auctions. And there are many pricing methods ,e.g., CPM (cost-per-mille), CPC (cost-per-click), CPA (cost-per-action), OCPM (optimized cost-per-mille) and so on. In this paper, we study the OCPM method (i.e., advertisers bid for conversions while pay per mille) under VCG auction. However, automatically bid in each view to maximize advertisers' conversions while still meet their target cost-per-conversion in feeds is difficult. To deal with these difficulties, we propose a reinforcement learning framework, i.e., RSDRL (ROI-sensitive distributional reinforcement learning). By making full use of the characteristics of auction rules which are missed by other methods, we design a reward function to surrogate conversion events and a bid generation method based on theoretical results. We also provide some theoretical results to guide hyperparameter tuning. Last, we validate RSDRL on a large industrial dataset with millions of auctions. Plenty of experiments (both online and offline) are used to evaluate the performance of our framework and RSDRL yields substantially better results than compared algorithms.


Optimized Cost per Mille in Feeds Advertising

The plan recognition task is to infer an actor’s plan and goal given observations about its behavior. We submit that in some cases, for plan recognition to be effective and complete, it must appeal to a notion of epistemics to i) recognize epistemic goals, where the actor is trying to achieve some state of knowledge or belief; and ii) model the observer, and its knowledge of the actor, as first class elements of the plan recognition process. To this end, we formalize the notion of Epistemic Plan Recognition, which builds on two growing areas of research: epistemic planning and plan recognition. Our epistemic plan recognition specification appeals to an epistemic logic framework to represent agent beliefs. To realize our specification, we cast the epistemic plan recognition problem as an epistemic planning problem, whose solutions can be generated using existing epistemic planning tools. Finally, we evaluate our approach by utilizing and comparing existing epistemic planners on a diverse set of epistemic plan recognition problems.


Epistemic Plan Recognition

This paper focuses on two-sided matching where one side (a hospital or firm) is matched to the other side (a doctor or worker) so as to maximize a cardinal objective under general feasibility constraints. In a standard model, even though multiple doctors can be matched to a single hospital, a hospital has a responsive preference and a maximum quota. However, in practical applications, a hospital has some complicated cardinal preference and constraints. With such preferences (e.g., submodular) and constraints (e.g., knapsack or matroid intersection), stable matchings may fail to exist. This paper first determines the complexity of checking and computing stable matchings based on preference class and constraint class. Second, we establish a framework to analyze this problem on packing problems, and the framework enables us to access the wealth of online packing algorithms so that we construct approximately stable algorithms as a variant of generalized deferred acceptance algorithm. We further provide some inapproximability results. 


Approximately Stable Matchings with General Constraints

Employing Models of Human Social Motor Behavior for Artifical Agent Trainers

We present a framework for verifying strategic behaviour in an unbounded multi-agent system. We introduce a novel probabilistic semantics for parameterised multi-agent systems and define the corresponding verification problem against two probabilistic variants of alternating-time temporal logic. We define a verification procedure using an abstract model construction. We show that the procedure is complete for one variant of our specification language, and partial for the other. We present an implementation and report experimental results. 


Parameterised Verification of Strategic Properties in Probabilistic Multi-Agent Systems

In computational social choice, shift bribery is the procedure of paying voters to shift the briber’s preferred candidate forward in their preferences so as to make this candidate an election winner; the more general swap bribery procedure also allows one to pay voters to swap other candidates in their preferences. The complexity of swap and shift bribery is well-understood for many voting rules; typically, finding a minimum-cost bribery is computationally hard. We have studied swap and shift bribery in the setting where voters’ preferences are known to be single-peaked or single-crossing. We obtain polynomial-time algorithms for several variants of these problems for classic voting rules, such as Plurality, Borda and Condorcet-consistent rules. In this talk we present a polynomial time algorithm for shift bribery in the setting where voters’ preferences are single-peaked. 
"


Algorithms for Swap and Shift Bribery in Structured Elections

Redistribution mechanisms have been proposed for more efficient resource allocation but not for profit. We consider redistribution mechanism design in a setting where participants are connected and the resource owner is only connected to some of them. In this setting, to make the resource allocation more efficient, the resource owner has to inform the others who are not her neighbours, but her neighbours do not want more participants to compete with them. Hence, the goal is to design a redistribution mechanism such that participants are incentivized to invite more participants and the resource owner does not earn or lose much money from the allocation. In this presentation, I will first show that existing redistribution mechanisms cannot be directly applied in the network setting. Then I will introduce a novel network-based redistribution mechanism such that all participants on the network are invited, the allocation is more efficient and the resource owner has no deficit.


Redistribution Mechanism on Networks

When planning for multi-robot navigation tasks under uncertainty, the presence of multiple robots in the same part of the environment simultaneously can have an adverse effect on navigation performance. We refer to this effect as congestion. Existing solutions fail to model congestion at planning time, which leads to inefficient behaviour. In this presentation, I will detail how to probabilistically model congestion, presenting a structure called the probabilistic reservation table that summarises the plans of robots using continuous-time Markov chains. I will then show how this structure can be used for planning by outlining an approach that, for each robot, sequentially builds and solves a Markov decision process where the transition probabilities are obtained by querying the probabilistic reservation table.


Multi-Robot Planning Under Uncertainty with Congestion-Aware Models

Sparse reward problems present a challenge for reinforcement learning (RL) agents. Previous work has shown that choosing start states according to a curriculum can significantly improve the learning performance. We observe that many existing curriculum generation algorithms rely on two key components: Performance measure estimation and a start selection policy. Therefore, we propose a unifying framework for performance-based start state curricula in RL, which allows to analyze and compare the performance influence of the two key components. Furthermore, a new start state selection policy using spatial performance measure gradients is introduced. We conduct extensive empirical evaluations to compare performance-based start state curricula and investigate the influence of performance measure model choice and estimation. Benchmarking on difficult robotic navigation tasks and a high-dimensional robotic manipulation task, we demonstrate state-of-the-art performance of our novel spatial gradient curriculum.

Downloads

Next from AAMAS 2020

Learning to Design Coupons in Online Advertising Markets

Similar lecture

Contract-based Design and Verification of Multi-Agent Systems with Quantitative Temporal Requirements

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Learning to Design Coupons in Online Advertising Markets

Similar lecture

Contract-based Design and Verification of Multi-Agent Systems with Quantitative Temporal Requirements

Downloads