Italy

Given a speech clip and facial image, the goal of talking face generation is to synthesize a talking face video with accurate mouth synchronization and natural face motion. Recent progress has proven the effectiveness of the landmarks as the intermediate information during talking face generation. However, the large gap between audio and visual modalities makes the prediction of landmarks challenging and limits generation ability. This paper proposes a semantic and temporal synchronous landmark learning method for talking face generation. First, we propose to introduce a word detector to enforce richer semantic information. Then, we propose to preserve the temporal synchronization and consistency between landmarks and audio via the proposed temporal residual loss. Lastly, we employ a U-Net generation network with adaptive reconstruction loss to generate facial images for the predicted landmarks. Experimental results on two benchmark datasets LRW and GRID demonstrate the effectiveness of our model compared to the state-of-the-art methods of talking face generation.

ICPR 2020

Talking face generation via Learning Semantic and Temporal Synchronous Landmarks

deep leaning

talking face generation

cross-modal

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

NAACL 2022

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

poster

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 



This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Single view learning in action recognition

Color is very important feature for image representation, it assumes very essential in the human visual recognition process. Although, several color extension of Local Binary Pattern processes the color texture from the three color channels separately (Marginal way). Aware of the high interaction that exists between different channels in the color image, this work introduces a new vector process to define a local descriptor for color images named Local Binary Quaternion Rotation Pattern (LBQRP). In this LBQRP purpose, the quaternion representation is used to represent color texture. The distance between two color can be expressed as the angle of rotation between two unit quaternions using the geodesic distance. After a LBQRP coding, local histograms are extracted and used as features. Experiments on three challenging color datasets: Vistex, Outex-TC13 and USPtex are carried out to evaluate the LBQRP performance in texture classification. Results show the high efficiency of the proposed approach facing to several stat-of-art methods.

Local Binary Quaternion Rotation Pattern for colour texture retrieval


Local binary pattern (LBP) is sensitive to inverse grayscale changes. In this paper, we propose first- and secondorder sorted LBP (SLBP) features which are robust to inverse grayscale changes and image rotation. The proposed method focuses on encoding difference-sign information and it can be generalized to embed other difference-magnitude features to obtain complementary representations. Experiments demonstrate the effectiveness of our method for texture classification under (linear or nonlinear) grayscale-inversion and rotation changes.

First- and Second-Order Sorted Local Binary Pattern Features for Grayscale-Inversion and Rotation Invariant Texture Classification


Recently, a novel radical analysis network (RAN) has the
capability of effectively recognizing unseen Chinese
character classes and largely reducing the requirement of
training data by treating a Chinese character as a
hierarchical composition of radicals rather than a single
character class.} However, when dealing with more
challenging issues, such as the recognition of complicated
characters, low-frequency character categories, and
characters in natural scenes, RAN still has a lot of room
for improvement. In this paper, we explore options to
further improve the structure generalization and robustness
capability of RAN with the Transformer architecture, which
has achieved start-of-the-art results for many
sequence-to-sequence tasks. More specifically, we propose
to replace the original attention module in RAN with the
transformer decoder, which is named as a transformer-based
radical analysis network (RTN). The experimental results
show that the proposed approach can significantly
outperform the RAN on both printed Chinese character
database and natural scene Chinese character database.
Meanwhile, further analysis proves that RTN can be better
generalized to complex samples and low-frequency
characters, and has better robustness in recognizing
Chinese characters with different attributes.

A Transformer-based Radical Analysis Attention Network for 
Chinese Character Recognition

This work explores the biases in learning processes based on deep neural network architectures. We analyze how bias affects deep learning processes through a toy example using the MNIST database and a case study in gender detection from face images. We employ two gender detection models based on popular deep neural networks. We present a comprehensive analysis of bias effects when using an unbalanced training dataset on the features learned by the models. We show how bias impacts in the activations of gender detection models based on face images. We finally propose InsideBias, a novel method to detect biased models. InsideBias is based on how the models represent the information instead of how they perform, which is the normal practice in other existing methods for bias detection. Our strategy with InsideBias allows to detect biased models with very few samples (only 15 images in our case study). Our experiments include 72K face images from 24K identities and 3 ethnic groups.

InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometrics

Image-text matching has made great progresses recently, but there still remains challenges in fine-grained matching. To deal with this problem, we propose an Improved Visual Semantic Reasoning model (VSR++), which jointly models 1)
global alignment between images and texts and 2) local correspondence between regions and words in a unified framework. To exploit their complementary advantages, we also develop a suitable learning strategy to balance their relative importance. As a result, our model can distinguish image regions and text words in a fine-grained level, and thus achieves the current state-of-the-art performance on two benchmark datasets.

VSR++: Improving Visual Semantic Reasoning for Fine-Grained Image-Text Matching

Current compressed video action recognition methods are mainly based on complete data. However, in a real transmission scenario, the compressed video packets are usually disorderly received and even lost due to network jitters or
congestion. To recognize actions in early phases with limited packets, e.g. for quickly forecasting possible potential risks, in this paper, we propose a Temporal Enhanced Multi-Stream Network (TEMSN) towards practical compressed video action
recognition. First, we make use of three modalities in the compressed domain as complementary cues and build a multistream network to capture rich information from compressed video packets. Second, we design a temporal enhanced module
based on an Encoder-Decoder structure, which is applied to each stream to infer missing packets, generating more accurate action dynamics. Thanks to the multiple modalities and their temporal enhancement, our approach better models actions with partial available compressed video packets. Experiments on the HMDB-51 and UCF-101 datasets validate its effectiveness and efficiency.

Towards Practical Compressed Video Action Recognition: A
Temporal Enhanced Multi-Stream Network


This paper strives for action recognition and detection in video modalities like RGB, depth maps or 3D-skeleton sequences when only limited modality-specific labeled examples
are available. For the RGB, and derived optical-flow, modality
many large-scale labeled datasets have been made available. They
have become the de facto pre-training choice when recognizing
or detecting new actions from RGB datasets that have limited
amounts of labeled examples available. Unfortunately, large-scale
labeled action datasets for other modalities are unavailable for
pre-training. In this paper, our goal is to recognize actions from
limited examples in non-RGB video modalities, by learning from
large-scale labeled RGB data. To this end, we propose a two-step
training process: (i) we extract action representation knowledge
from an RGB-trained teacher network and adapt it to a nonRGB student network. (ii) we then fine-tune the transfer model
with available labeled examples of the target modality. For the
knowledge transfer we introduce feature-supervision strategies,
which rely on unlabeled pairs of two modalities (the RGB and
the target modality) to transfer feature level representations from
the teacher to the student network. Ablations and generalizations
with two RGB source datasets and two non-RGB target datasets
demonstrate that an optical-flow teacher provides better action
transfer features than RGB for both depth maps and 3Dskeletons, even when evaluated on a different target domain, or
for a different task. Compared to alternative cross-modal action
transfer methods we show a good improvement in performance
especially when labeled non-RGB examples to learn from are
scarce.

Feature-Supervised Action Modality Transfer

Feature point matching is an important problem because its applications cover a wide range of tasks in computer vision. Deep learning-based methods for learning local features have recently shown superior performance. However, it is not easy to collect the training data in these methods, especially in cross-spectral settings such as the correspondence between RGB and near-infrared images. In this paper, we propose an unsupervised learning method for general feature point matching. Because we train a convolutional neural network as a feature extractor in order to satisfy the cycle consistency of the correspondences between an input image pair, the proposed method does not require supervision and works even in cross-spectral settings. In our experiments, we apply the proposed method to stereo matching, which is a dense feature point matching problem. The experimental results, which simulate cross-spectral settings with three different settings, i.e., RGB stereo, RGB vs gray-scale, and anaglyph (red vs cyan), show that our proposed method outperforms the compared methods, which employ handcrafted features for stereo matching, by a significant margin.

Feature Point Matching in Cross-Spectral Images with Cycle Consistency Learning

Generative Adversarial Network (GAN) has been widely used for object attribute editing. However, the semantic correlation, resulted from the feature map interaction in the generative network of GAN, may impair the generalization ability of the generative network. In this work, semantic disentanglement is introduced in GAN to reduce the attribute correlation. The feature maps of the generative network are first grouped with an efficient clustering algorithm based on hash encoding, which are used to excavate hidden semantic attributes and calculate
the group-wise orthogonality loss for the reduction of attribute entanglement.
Meanwhile, the feature maps falling in the intersection regions of different groups are further suppressed to reduce the attribute-wise interaction.
Extensive experiments reveal that the proposed GAN generated more genuine objects than the state of the arts. Quantitative results of classification accuracy, inception score and FID score further justify the effectiveness of the proposed GAN.

Premium content

Talking face generation via Learning Semantic and Temporal Synchronous Landmarks

Downloads

Next from ICPR 2020

Single view learning in action recognition

Similar lecture

Cross-Lingual Cross-Modal Consolidation for Effective Multilingual Video Corpus Moment Retrieval