United States

Neural vocoders based on the generative adversarial neural network (GAN) have been widely used due to their fast inference speed and lightweight networks while generating high-quality speech waveforms. Since the perceptually important speech components are primarily concentrated in the low-frequency bands, most GAN-based vocoders perform multi-scale analysis that evaluates downsampled speech waveforms. This multi-scale analysis helps the generator improve speech intelligibility. However, in preliminary experiments, we discovered that the multi-scale analysis which focuses on the low-frequency bands causes unintended artifacts, e.g., aliasing and imaging artifacts, which degrade the synthesized speech waveform quality. Therefore, in this paper, we investigate the relationship between these artifacts and GAN-based vocoders and propose a GAN-based vocoder, called Avocodo, that allows the synthesis of high-fidelity speech with reduced artifacts. We introduce two kinds of discriminators to evaluate speech waveforms in various perspectives: a collaborative multi-band discriminator and a sub-band discriminator. We also utilize a pseudo quadrature mirror filter bank to obtain downsampled multi-band speech waveforms while avoiding aliasing. According to experimental resutls, Avocodo outperforms baseline GAN-based vocoders, both objectviely and subjectively, while reproducing speech with fewer artifacts.

AAAI 2023

Avocodo: Generative Adversarial Network for Artifact-free Vocoder

other foundations of speech & natural language processing

speech and multimodality

generation

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

poster

Video copy localization aims to precisely localize all the copied segments within a pair of untrimmed videos in video retrieval applications. Previous methods typically start from frame-to-frame similarity matrix generated by cosine similarity between frame-level features of the input video pair, and then detect and refine the boundaries of copied segments on similarity matrix under temporal constraints. In this paper, we propose TransVCL: an attention-enhanced video copy localization network, which is optimized directly from initial frame-level features and trained end-to-end with three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for similarity matrix generation, and a temporal alignment module for copied segments localization. In contrast to previous methods demanding the handcrafted similarity matrix, TransVCL incorporates long-range temporal information between feature sequence pair using self- and cross- attention layers. With the joint design and optimization of three components, the similarity matrix can be learned to present more discriminative copied patterns, leading to significant improvements over previous methods on segment-level labeled datasets (VCSL and VCDB). Besides the state-of-the-art performance in fully supervised setting, the attention architecture facilitates TransVCL to further exploit unlabeled or simply video-level labeled data. Additional experiments of supplementing video-level labeled datasets including SVD and FIVR reveal the high flexibility of TransVCL from full supervision to semi-supervision (with or without video-level annotation). Code is publicly available at https://github.com/transvcl/TransVCL.

TransVCL: Attention-enhanced Video Copy Localization Network with Flexible Supervision

Active learning (AL) aims to find a better trade-off between labeling costs and model performance by consciously selecting more informative samples to label. Recently, adversarial approaches have emerged as an effective solution to AL. Most of them leverage Generative Adversarial Networks (GAN) to align the feature distribution of labeled and unlabeled data, upon which discriminators are trained to better distinguish between them. However, these methods fail to consider the relationship between unlabeled samples and decision boundaries. Consequently, they tend to generate confused features near class boundaries, thus decreasing the ultimate performance. To address this, this paper proposes a novel adversarial AL method, namely Multi-classifier Adversarial Optimization for Active Learning (MAOAL), which trains multiple classifiers in an adversarial manner. MAOAL employs task-specific decision boundaries for data alignment while selecting the most informative samples to label. To fulfill this, we introduce a novel classifier class confusion (C3) metric, which represents the classifier discrepancy as the interclass correlation of classifier outputs. Without any additional hyper-parameters, the C3 metric further reduces the negative impacts of ambiguous samples in the process of distribution alignment and sample selection. More concretely, the network is trained adversarially by adding two auxiliary classifiers, reducing the distribution bias of labeled and unlabeled samples by minimizing the C3 loss between classifiers, while learning tighter decision boundaries and highlighting hard samples by maximizing the C3 loss. Finally, the unlabeled samples with the highest C3 loss are selected for labeling. Extensive experimental results demonstrate the superiority of our approach over state-of-the-art AL methods in terms of both image classification and object detection tasks.



Multi-classifier Adversarial Optimization for¬†Active Learning

In this paper, we study the problem of MOOC quality evaluation that is essential for improving the course materials, promoting students' learning efficiency, and benefiting user services. 
While achieving promising performances, current works still suffer from the complicated interactions and relationships of entities in MOOC platforms. 
To tackle the challenges, we formulate the problem as a course representation learning task based, and develop an Information-aware Graph Representation Learning(IaGRL) for multi-view MOOC quality evaluation. 
Specifically, We first build a MOOC Heterogeneous Network (HIN) to represent the interactions and relationships among entities in MOOC platforms. 
And then we decompose the MOOC HIN into multiple single-relation graphs based on meta-paths to depict multi-view semantics of courses. 
The course representation learning can be further converted to a multi-view graph representation task. 
Different from traditional graph representation learning, the learned course representations are expected to match the following three types of validity: 
(1) the agreement on expressiveness between the raw course portfolio and the learned course representations; 
(2) the consistency between the representations in each view and the unified representations; 
(3) the alignment between the course and MOOC platform representations. 
Therefore, we propose to exploit mutual information for preserving the validity of course representations. 
We conduct extensive experiments over real-world MOOC datasets to demonstrate the effectiveness of our proposed method.


Multi-View MOOC Quality Evaluation via Information-Aware Graph Representation Learning

In this work, we are dedicated to leveraging the BERT pre-training success and modeling the domain-specific statistics to fertilize the sign language recognition~(SLR) model. Considering the dominance of hand and body in sign language expression, we organize them as pose triplet units and feed them into the Transformer backbone in a frame-wise manner. Pre-training is performed via reconstructing the masked triplet unit from the corrupted input sequence, which learns the hierarchical correlation context cues among internal and external triplet units. Notably, different from the highly semantic word token in BERT, the pose unit is a low-level signal originally locating in continuous space, which prevents the direct adoption of the BERT cross entropy objective. To this end, we bridge this semantic gap via coupling tokenization of the triplet unit. It adaptively extracts the discrete pseudo label from the pose triplet unit, which represents the semantic gesture / body state. After pre-training, we fine-tune the pre-trained encoder on the downstream SLR task, jointly with the newly added task-specific layer. Extensive experiments are conducted to validate the effectiveness of our proposed method, achieving new state-of-the-art performance on all four benchmarks with a notable gain.

BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization

In robust Markov decision processes (MDPs), the uncertainty in the transition kernel is addressed by finding a policy that optimizes the worst-case performance over an uncertainty set of MDPs. While much of the literature has focused on discounted MDPs, robust average-reward MDPs remain largely unexplored. In this paper, we focus on robust average-reward MDPs, where the goal is to find a policy that optimizes the worst-case average reward over an uncertainty set. We first take an approach that approximates average-reward MDPs using discounted MDPs. We prove that the robust discounted value function converges to the robust average-reward as the discount factor $\gamma$ goes to $1$, and moreover, when $\gamma$ is large, any optimal policy of the robust discounted
MDP is also an optimal policy of the robust average-reward. We further design a robust dynamic programming approach, and theoretically characterize its convergence to the optimum. Then, we investigate robust average-reward MDPs directly without using discounted MDPs as an intermediate step. We derive the robust Bellman equation for robust average-reward MDPs, prove that the optimal policy can be derived from its solution, and further design a robust relative value iteration algorithm that provably find its solution, or equivalently, the optimal robust policy. 

Robust Average-Reward Markov Decision Processes

1- Introduction
Food information engineering involves the acquisition, the processing, the diffusion and the use of food information to different stakeholders. These information are compiled from several data sources and used for a variety of purposes such as Question Answering, food image recognition, food recommendation, etc. Many authors have proposed AI technologies for the acquisition, the processing, the diffusion and the use of food information. However, these contributions are scattered in many scientific papers and are difficult to exploit. The goal of this research is to provide to the research community with: (1) a set of resources that will allow them to document this domain and (2) a body of knowledge that will help to reduce the research curve.

2- Research methodology
The research methodology consists of an iterative and incremental process during which Open Research Knowledge Graph (ORKG - orkg.org) is used to create/update templates (used to register researchers contributions), comparison tables of papers and SmartReviews. The idea is to use the experience gained during the creation, use and update of these resources to improve them and create new ones. It was divided into two phases: an exploratory phase and a systematic literature review phase.

The exploratory phase took place during a six months (June - November 2022) curation of ORKG with the help of the ORKG team. Thus, we benefit from the training of the ORKG team. The curation meetings allowed us to benefit from the remarks of the other curators and the ORKG team. The overall work was evaluated during this curation. The exploratory phase allowed us to have a global overview of the domain and to create a set of templates and comparisons tables. The systematic literature review phase consists of using these resources to compare more papers and update existing comparisons with new contributions.

3- Results
Globally, during the exploratory phase, more than 400 contributions were extracted from 120 papers and ingested in the ORKG platform using 10 templates that we created/updated or that were already created. Using these contributions, 33 comparison tables of papers and 2 SmartReviews were created. The overall results of this work can be found on the ORKG platform: (https://orkg.org/author/R138055).

Firstly, 5 templates for documenting food composition tables, food ontologies and knowledge graph construction, description and integration were created. Thereafter, we used these templates to review the work in these domains. The comparison tables allowed us to identify structured, unstructured and semi-structured sources used to acquire food knowledge and build food ontologies, food knowledge graphs and food composition tables. On the other hand, we found that Neural Networks were proposed for calorie prediction from food images; knowledge was manually, semi-automatically and automatically extracted from these data sources. Concerning the automatic acquisition of knowledge from data sources, we found that rule-based approaches, machine learning, deep learning and natural language processing techniques were proposed. We also found that on top of food knowledge graphs, Question Answering systems are generally built. Thus, we decided to document it. Other templates for documenting food Question Answering and benchmark were created.

The second phase of this work is ongoing and it consists of a systematic literature review of food information engineering. On the other hand, even if Neurosymbolic AI have shown promising results in many domains, we found that this approach is not yet used in the domain of food information engineering. This research direction has to be considered.

Food Information Engineering: a systematic literature review

Algorithms are ubiquitous: they track our sleep, help us find cheap flights, and even help us see black holes. However, designing novel algorithms is extremely difficult, and we do not have efficient algorithms for many fundamental problems. The goal of my research is to accelerate algorithm discovery by building an automatic computer scientist. To work towards this goal, my research focuses on inductive logic programming, a form of machine learning in which my collaborators and I have demonstrated major advances in automated algorithm discovery over the past five years.

The Automatic Computer Scientist

In this position paper, we propose a new approach to generating a type of knowledge base (KB) from text, based on question generation and entity linking. We argue that the proposed type of KB has many of the key advantages of a traditional symbolic KB: in particular, it consists of small modular components, which can be combined compositionally to answer complex queries, including relational queries and queries involving “multi-hop” inferences. However, unlike a traditional KB, this information store is well-aligned with common user information needs.

QA is the New KR: Question-Answer Pairs as Knowledge Bases

There is a fast-growing literature in addressing the fairness of
AI models (fair-AI), with a continuous stream of new conceptual
frameworks, methods, and tools. How much can we trust
them? How much do they actually impact society? We take
a critical focus on fair-AI and survey issues, simplifications,
and mistakes that researchers and practitioners often underestimate,
which in turn can undermine the trust on fair-AI
and limit its contribution to society. In particular, we discuss
the hyper-focus on fairness metrics and on optimizing their
average performances. We instantiate this observation by discussing
the Yule’s effect of fair-AI tools: being fair on average
does not imply being fair in contexts that matter. We conclude
that the use of fair-AI methods should be complemented with
the design, development, and verification practices that are
commonly summarized under the umbrella of trustworthy AI.

Can We Trust Fair-AI?

Adversarial robustness studies the worst-case performance
of a machine learning model to ensure safety and reliability. With the proliferation of deep-learning-based technology, the potential risks associated with model development and deployment can be amplified and become dreadful vulnerabilities. This paper provides a comprehensive overview of research topics and foundational principles of research methods for adversarial robustness of deep learning models, including attacks, defenses, verification, and novel applications.

Avocodo: Generative Adversarial Network for Artifact-free Vocoder

Downloads

Next from AAAI 2023

TransVCL: Attention-enhanced Video Copy Localization Network with Flexible Supervision

Similar lecture

Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data