United States

The problem of document structure reconstruction refers to converting digital or scanned documents into corresponding semantic structures. Most existing works mainly focus on splitting the boundary of each element in a single document page, neglecting the reconstruction of semantic structure in multi-page documents. This paper introduces hierarchical reconstruction of document structures as a novel task suitable for NLP and CV fields. To better evaluate the system performance on the new task, we built a large-scale dataset named HRDoc, which consists of 2,500 multi-page documents with nearly 2 million semantic units. Every document in HRDoc has line-level annotations including categories and relations obtained from rule-based extractors and human annotators. Moreover, we proposed an encoder-decoder-based hierarchical document structure parsing system (DSPS) to tackle this problem. By adopting a multi-modal bidirectional encoder and a structure-aware GRU decoder with soft-mask operation, the DSPS model surpass the baseline method by a large margin. All scripts and datasets will be made publicly available at https://github.com/jfma-USTC/HRDoc.

AAAI 2023

HRDoc: Dataset and Baseline Method Toward Hierarchical Reconstruction of Document Structures

snlp: applications

cv: language and vision

cv: applications

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

poster

Automatic image cropping algorithms aim to recompose images like human-being photographers by generating the cropping boxes with improved composition quality. Cropping box regression approaches learn the beauty of composition from annotated cropping boxes. However, the bias of annotations leads to quasi-trivial recomposing results, which has an obvious tendency to the average location of training samples. The crux of this predicament is that the task is naively treated as a box regression problem, where rare samples might be dominated by normal samples, and the composition patterns of rare samples are not well exploited. Observing that similar composition patterns tend to be shared by the cropping boundaries annotated nearly, we argue to find the beauty of composition from the rare samples by clustering the samples with similar cropping boundary annotations, i.e., similar composition patterns. We propose a novel Contrastive Composition Clustering (C2C) to regularize the composition features by contrasting dynamically established similar and dissimilar pairs. In this way, common composition patterns of multiple images can be better summarized, which especially benefits the rare samples and endows our model with better generalizability to render nontrivial results. Extensive experimental results show the superiority of our model compared with prior arts. We also illustrate the philosophy of our design with an interesting analytical visualization.

Find Beauty in the Rare: Contrastive Composition Feature Clustering for Nontrivial Cropping Box Regression

Continual Test-Time Adaptation (CTTA) aims to adapt the source model to continually changing unlabeled target domains without access to the source data. Existing methods mainly focus on model-based adaptation in a self-training manner, such as predicting pseudo labels for new domain datasets. Since pseudo labels are noisy and unreliable, these methods suffer from catastrophic forgetting and error accumulation when dealing with dynamic data distributions. Motivated by the prompt learning in NLP, in this paper, we propose to learn an image-layer visual domain prompt for target domains while having the source model parameters frozen. During testing, the changing target datasets can be adapted to the source model by reformulating the input data with the learned visual prompts. Specifically, we devise two types of prompts, i.e., domains-specific prompts and domains-agnostic prompts, to extract current domain knowledge and maintain the domain-shared knowledge in the continual adaptation. Furthermore, we design a homeostasis-based adaptation strategy to suppress domain-sensitive parameters in domain-invariant prompts to learn domain-shared knowledge more effectively. This transition from the model-dependent paradigm to the model-free one enables us to bypass the catastrophic forgetting and error accumulation problems. Experiments show that our proposed method achieves significant performance gains over state-of-the-art methods on four widely-used benchmarks, including CIFAR-10C, CIFAR-100C, ImageNet-C, and VLCS datasets.

Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation

Graph neural networks (GNNs) have shown their superiority in modeling graph data. Owing to the advantages of federated learning, federated graph learning (FGL) enables clients to train strong GNN models in a distributed manner without sharing their private data. A core challenge in federated systems is the non-IID problem, which also widely exists in real-world graph data. For example, local data of clients may come from diverse datasets or even domains, e.g., social networks and molecules, increasing the difficulty for FGL methods to capture commonly shared knowledge and learn a generalized encoder. From real-world graph datasets, we observe that some structural properties are shared by various domains, presenting great potential for sharing structural knowledge in FGL. Inspired by this, we propose FedStar, an FGL framework that extracts and shares the common underlying structure information for inter-graph federated learning tasks. To explicitly extract the structure information rather than encoding them along with the node features, we define structure embeddings and encode them with an independent structure encoder. Then, the structure encoder is shared across clients while the feature-based knowledge is learned in a personalized way, making FedStar capable of capturing more structure-based domain-invariant information and avoiding feature misalignment issues. We perform extensive experiments over both cross-dataset and cross-domain non-IID FGL settings, demonstrating the superiority of FedStar.

Federated Learning on Non-IID Graphs via Structural Knowledge Sharing

Neural vocoders based on the generative adversarial neural network (GAN) have been widely used due to their fast inference speed and lightweight networks while generating high-quality speech waveforms. Since the perceptually important speech components are primarily concentrated in the low-frequency bands, most GAN-based vocoders perform multi-scale analysis that evaluates downsampled speech waveforms. This multi-scale analysis helps the generator improve speech intelligibility. However, in preliminary experiments, we discovered that the multi-scale analysis which focuses on the low-frequency bands causes unintended artifacts, e.g., aliasing and imaging artifacts, which degrade the synthesized speech waveform quality. Therefore, in this paper, we investigate the relationship between these artifacts and GAN-based vocoders and propose a GAN-based vocoder, called Avocodo, that allows the synthesis of high-fidelity speech with reduced artifacts. We introduce two kinds of discriminators to evaluate speech waveforms in various perspectives: a collaborative multi-band discriminator and a sub-band discriminator. We also utilize a pseudo quadrature mirror filter bank to obtain downsampled multi-band speech waveforms while avoiding aliasing. According to experimental resutls, Avocodo outperforms baseline GAN-based vocoders, both objectviely and subjectively, while reproducing speech with fewer artifacts.

Avocodo: Generative Adversarial Network for Artifact-free Vocoder

In robust Markov decision processes (MDPs), the uncertainty in the transition kernel is addressed by finding a policy that optimizes the worst-case performance over an uncertainty set of MDPs. While much of the literature has focused on discounted MDPs, robust average-reward MDPs remain largely unexplored. In this paper, we focus on robust average-reward MDPs, where the goal is to find a policy that optimizes the worst-case average reward over an uncertainty set. We first take an approach that approximates average-reward MDPs using discounted MDPs. We prove that the robust discounted value function converges to the robust average-reward as the discount factor $\gamma$ goes to $1$, and moreover, when $\gamma$ is large, any optimal policy of the robust discounted
MDP is also an optimal policy of the robust average-reward. We further design a robust dynamic programming approach, and theoretically characterize its convergence to the optimum. Then, we investigate robust average-reward MDPs directly without using discounted MDPs as an intermediate step. We derive the robust Bellman equation for robust average-reward MDPs, prove that the optimal policy can be derived from its solution, and further design a robust relative value iteration algorithm that provably find its solution, or equivalently, the optimal robust policy. 

Robust Average-Reward Markov Decision Processes

We present nBIIG, a neural Business Intelligence (BI) Insights Generation system. Given a table, our system applies various analyses to create corresponding RDF representations, and then uses a neural model to generate fluent textual insights out of these representations. The generated insights can be used by an analyst, via a human-in-the-loop paradigm, to enhance the task of creating compelling table reports. The underlying generative neural model is trained over large and carefully distilled data, curated from multiple BI domains. Thus, the system can generate faithful and fluent insights over open-domain tables, making it practical and useful.

nBIIG: A Neural BI Insights Generation System for Table Reporting

1- Introduction
Food information engineering involves the acquisition, the processing, the diffusion and the use of food information to different stakeholders. These information are compiled from several data sources and used for a variety of purposes such as Question Answering, food image recognition, food recommendation, etc. Many authors have proposed AI technologies for the acquisition, the processing, the diffusion and the use of food information. However, these contributions are scattered in many scientific papers and are difficult to exploit. The goal of this research is to provide to the research community with: (1) a set of resources that will allow them to document this domain and (2) a body of knowledge that will help to reduce the research curve.

2- Research methodology
The research methodology consists of an iterative and incremental process during which Open Research Knowledge Graph (ORKG - orkg.org) is used to create/update templates (used to register researchers contributions), comparison tables of papers and SmartReviews. The idea is to use the experience gained during the creation, use and update of these resources to improve them and create new ones. It was divided into two phases: an exploratory phase and a systematic literature review phase.

The exploratory phase took place during a six months (June - November 2022) curation of ORKG with the help of the ORKG team. Thus, we benefit from the training of the ORKG team. The curation meetings allowed us to benefit from the remarks of the other curators and the ORKG team. The overall work was evaluated during this curation. The exploratory phase allowed us to have a global overview of the domain and to create a set of templates and comparisons tables. The systematic literature review phase consists of using these resources to compare more papers and update existing comparisons with new contributions.

3- Results
Globally, during the exploratory phase, more than 400 contributions were extracted from 120 papers and ingested in the ORKG platform using 10 templates that we created/updated or that were already created. Using these contributions, 33 comparison tables of papers and 2 SmartReviews were created. The overall results of this work can be found on the ORKG platform: (https://orkg.org/author/R138055).

Firstly, 5 templates for documenting food composition tables, food ontologies and knowledge graph construction, description and integration were created. Thereafter, we used these templates to review the work in these domains. The comparison tables allowed us to identify structured, unstructured and semi-structured sources used to acquire food knowledge and build food ontologies, food knowledge graphs and food composition tables. On the other hand, we found that Neural Networks were proposed for calorie prediction from food images; knowledge was manually, semi-automatically and automatically extracted from these data sources. Concerning the automatic acquisition of knowledge from data sources, we found that rule-based approaches, machine learning, deep learning and natural language processing techniques were proposed. We also found that on top of food knowledge graphs, Question Answering systems are generally built. Thus, we decided to document it. Other templates for documenting food Question Answering and benchmark were created.

The second phase of this work is ongoing and it consists of a systematic literature review of food information engineering. On the other hand, even if Neurosymbolic AI have shown promising results in many domains, we found that this approach is not yet used in the domain of food information engineering. This research direction has to be considered.

Food Information Engineering: a systematic literature review

Algorithms are ubiquitous: they track our sleep, help us find cheap flights, and even help us see black holes. However, designing novel algorithms is extremely difficult, and we do not have efficient algorithms for many fundamental problems. The goal of my research is to accelerate algorithm discovery by building an automatic computer scientist. To work towards this goal, my research focuses on inductive logic programming, a form of machine learning in which my collaborators and I have demonstrated major advances in automated algorithm discovery over the past five years.

The Automatic Computer Scientist

In this position paper, we propose a new approach to generating a type of knowledge base (KB) from text, based on question generation and entity linking. We argue that the proposed type of KB has many of the key advantages of a traditional symbolic KB: in particular, it consists of small modular components, which can be combined compositionally to answer complex queries, including relational queries and queries involving “multi-hop” inferences. However, unlike a traditional KB, this information store is well-aligned with common user information needs.

QA is the New KR: Question-Answer Pairs as Knowledge Bases

There is a fast-growing literature in addressing the fairness of
AI models (fair-AI), with a continuous stream of new conceptual
frameworks, methods, and tools. How much can we trust
them? How much do they actually impact society? We take
a critical focus on fair-AI and survey issues, simplifications,
and mistakes that researchers and practitioners often underestimate,
which in turn can undermine the trust on fair-AI
and limit its contribution to society. In particular, we discuss
the hyper-focus on fairness metrics and on optimizing their
average performances. We instantiate this observation by discussing
the Yule’s effect of fair-AI tools: being fair on average
does not imply being fair in contexts that matter. We conclude
that the use of fair-AI methods should be complemented with
the design, development, and verification practices that are
commonly summarized under the umbrella of trustworthy AI.

Premium content

HRDoc: Dataset and Baseline Method Toward Hierarchical Reconstruction of Document Structures

Downloads

Next from AAAI 2023

Find Beauty in the Rare: Contrastive Composition Feature Clustering for Nontrivial Cropping Box Regression

Similar lecture

HRDoc: Dataset and Baseline Method Toward Hierarchical Reconstruction of Document Structures