Brazil

Stochastic Value Gradient (SVG) methods underlie many recent achievements of model-based Reinforcement Learning agents in continuous state-action spaces. Despite their practical significance, many algorithm design choices still lack rigorous theoretical or empirical justification. In this work, we analyze one such design choice: the gradient estimator formula. We conduct our analysis on randomized Linear Quadratic Gaussian environments, allowing us to empirically assess gradient estimation quality relative to the actual SVG. Our results justify a widely used gradient estimator by showing it induces a favorable bias-variance tradeoff, which could explain the lower sample complexity of recent SVG methods.

BRACIS 2021

Gradient Estimation in Model-Based Reinforcement Learning: a Study on Linear Quadratic Environments

reinforcement learning; machine learning; stochastic optimization

2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

**Whova App** 
Stay in touch with your fellow conference attendees via the [Whova App](https://whova.com/portal/webapp/nacon_202106/)

**Conference Structure**
https://2021.naacl.org/blog/conference-structure/

**Walkthrough video of how to NAACL 2021** 

Please take a moment to view this video explaining how to navigate the platform, attend sessions network with other attendees. 


<figure class="video_container">
  <iframe src="https://screencast-o-matic.com/watch/crhwbGVh3vx?v=6&ff=1&title=0&controls=1" width=640  height=350  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

<br>
<br>

NAACL 2021

technical paper

**BRACIS** originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – **SBIA** (22 editions), and the Brazilian Symposium on Neural Networks – **SBRN** (13 editions). <br>**ENIAC** is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. <br>**STIL** is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: **CE-IA**, **CE-IC** and **CE-PLN**.<br><br><br>

To access the 10th Brazilian Conference on Intelligent Systems (**BRACIS**), you need to register for the conference.  Please visit the Conference official site to register

[Register now](http://c4ai.inova.usp.br/bracis2021/)

Register now

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – SBIA (22 editions), and the Brazilian Symposium on Neural Networks – SBRN (13 editions). 
ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. 
STIL is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: CE-IA, CE-IC and CE-PLN

Task-oriented dialogue systems are very important due to their wide range of applications. In particular, those tasks that involve multiple domains have gained increasing attention in recent years, as the actual tasks performed by virtual assistants often span multiple domains. In this work, we propose the Divide-and-Conquer Distributed Architecture with Slot Sharing Mechanism (DCDA-S2M) system, which includes not only a distributed system aimed at managing dialogues through the pipeline architecture, but also a slot sharing mechanism that allows the system to obtain information during a conversation in one domain and reuse it in another domain, in order to avoid redundant interactions and make the dialogue more efficient. Results show that the distributed architecture outperforms the centralized one by 21.51% and that the slot sharing mechanism improves the system performance both in the success rate and in the number of turns during the dialogue, demonstrating that it can prevent the agent from requesting redundant information.

Slot Sharing Mechanism in Multi-Domain Dialogue Systems

The recent progress in Reinforcement Learning applications to Resource Management presents MDPs without a deeper analysis of the impacts of design decisions on agent performance. In this paper, we compare and contrast four different MDP formulations, discussing their computational requirements and impacts on agent performance by means of an empirical analysis. We conclude by showing that, when used with Multi-Layer Perceptrons, image-like formulations are more helpful when shorter jobs dominate in the data set, while compact formulations are more helpful when longer jobs are used, and when transfer learning is desired as well.

On the impact of MDP design for Reinforcement Learning agents in Resource Management

The coronavirus disease (COVID-19) pandemic has brought significant challenges worldwide through the consequences of increasing demand for the Intensive Care Unit (ICU) resources. This work presents the Multi-Agent System for Glycemic Control (MAS4GC) to assist health professionals leading with critical patients in the ICU. More specifically, the MAS4GC manages patients’ blood glucose through glycemic predictions, treatment, and monitoring recommendations to health professionals. Prediction models are applied to monitor patients’ blood glucose allowing health professionals to carry out preventive treatments. The glycemic control is included in the FAST HUG mnemonic to remember the key issues in the supportive care of critically ill patients. The MAS4GC methodological development process is presented with Tropos modeling, architectural design, and implementation with the PADE framework. Agents’ inference mechanisms are based on production rules defined by intensive care physician specialists applying their knowledge to indicate treatments for patients. Two experiments using patients with synthetic data were conducted to evaluate the results of the MAS4GC: (1) the prediction model achieved 90\% accuracy in blood glucose predictions for the next 4 hours, (2) 84\% similarity of treatment recommendations compared to a human specialist, and 78\% in recommendations for monitoring glycemic of critical patients.

Premium content

Gradient Estimation in Model-Based Reinforcement Learning: a Study on Linear Quadratic Environments

Downloads

Next from BRACIS 2021

Slot Sharing Mechanism in Multi-Domain Dialogue Systems

Similar lecture

Benchmarking Commercial Intent Detection Services with Practice-Driven Evaluations