United States

Contrastive learning is a form of distance learning that aims to learn invariant features from two related representations. In this paper, we explore the bold hypothesis that an image and its caption can be simply regarded as two different views of the underlying mutual information, and train a model to learn a unified vision-language representation space that encodes both modalities at once in a modality-agnostic manner. We first identify difficulties in learning a generic one-tower model for vision-language pretraining (VLP), and propose OneR as a simple yet effective framework for our goal. We discover intriguing properties that distinguish OneR from the previous works that learn modality-specific representation spaces such as zero-shot object localization, text-guided visual reasoning and multi-modal retrieval, and present analyses to provide insights into this new form of multi-modal representation learning. Thorough evaluations demonstrate the potential of a unified modality-agnostic VLP framework.

AAAI 2023

Unifying Vision-Language Representation Space with Single-tower Transformer

representation learning

machine learning

language and vision

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

poster

The increased integration of artificial intelligence (AI) technologies in human workflows has resulted in a new paradigm of AI-assisted decision making,
in which an AI model provides decision recommendations while humans make the final decisions. To best support humans in decision making, it is critical to obtain a quantitative understanding of how humans interact with and rely on AI. Previous studies often model humans' reliance on AI as an analytical process, i.e., reliance decisions are made based on cost-benefit analysis. However, theoretical models in psychology suggest that the reliance decisions can often be driven by {\em emotions} like humans' {\em trust} in AI models. In this paper, we propose a {\em hidden Markov model} to capture the affective process underlying the human-AI interaction in AI-assisted decision making, by characterizing how decision makers adjust their trust in AI over time and make reliance decisions based on their trust. Evaluations on real human behavior data collected from human-subject experiments show that the proposed model outperforms various baselines in accurately predicting humans' reliance behavior in AI-assisted decision making. Based on the proposed model, we further provide insights into how humans' trust and reliance dynamics in AI-assisted decision making is influenced by contextual factors like decision stakes and their interaction experiences. 

Modeling Human Trust and Reliance in AI-assisted Decision Making: A Markovian Approach

Weisfeiler-Lehman (WL) test is widely applied as a basic algorithm for graph machine learning models, such as graph kernels, graph metrics, and graph neural networks. However, its ability to describe structural information is insufficient because it focuses only on the consistency of graphs. Therefore, this limits the performance of existing methods based on the WL test. For example, the similarity and distance between graphs defined using the WL test are coarse measures that do not accurately describe the differences in structure. This paper clarifies these facts and proposes a novel graph metric called Wasserstein WL subtree (WWLS) distance. First, we adopt the WL subtree as the structural information of the node neighborhood. Then, we define the node metric using L1-approximated tree edit distance (L1-TED) between WL subtrees of nodes. Finally, we define the WWLS by combining the Wasserstein distance and the L1-TED. The WWLS can capture slight structural differences that are difficult with conventional metrics. We demonstrate its superior performance in metric validation and graph classification experiments.

Wasserstein Graph Distance based on L1‚ÄìApproximated Tree Edit Distance between Weisfeiler‚ÄìLehman Subtrees

This paper presents a rewriting method for Boolean circuits that minimizes small subcircuits with exact synthesis. Individual synthesis tasks are encoded as Quantified Boolean Formulas (QBFs) that capture the full flexibility for the implementation of multi-output subcircuits.
This is in contrast to SAT-based resynthesis, where "don't cares" are computed for an individual gate, and replacements are confined to circuitry that is used exclusively by that gate.
An implementation of our method achieved substantial size reductions compared to state-of-the-art methods across a wide range of benchmark circuits.

Circuit Minimization with QBF-Based Exact Synthesis

This paper addresses the issue of adversarial attacks on ethical AI systems. We investigate using moral axioms and rules of deontic logic in a norm learning framework to mitigate adversarial norm training. This model of moral intuition and construction provides AI systems with moral guard rails yet still allows for learning conventions. We evaluate our approach by drawing inspiration from a study commonly used in moral development research. This questionnaire aims to test an agent's ability to reason to moral conclusions despite opposed testimony. Our findings suggest that our model can still correctly evaluate moral situations and learn conventions in an adversarial training environment. We conclude that adding axiomatic moral prohibitions and deontic inference rules to a norm learning model makes it less vulnerable to adversarial attacks.

Mitigating Adversarial Norm Training with Moral Axioms

Bayesian neural networks (BNNs) retain NN structures with a probability distribution placed over their weights. With the introduced uncertainties and redundancies, BNNs are proper choices of robust controllers for safety-critical control systems. This paper considers the problem of verifying the safety of nonlinear closed-loop systems with BNN controllers over unbounded-time horizon. In essence, we compute a safe weight set such that as long as the BNN controller is always applied with weights sampled from the safe weight set, the controlled system is guaranteed to be safe. We propose a novel two-phase method for the safe weight set computation. First, we construct a reference safe control set that constraints the control inputs, through polynomial approximation to the BNN controller followed by polynomial-optimization-based barrier certificate generation. Then, the computation of safe weight set is reduced to a range inclusion problem of the BNN on the system domain w.r.t. the safe control set, which can be solved incrementally and the set of safe weights can be extracted. Compared with the existing method based on invariant learning and mixed-integer linear programming, we could compute safe weight sets with larger radii on a series of linear benchmarks. Moreover, experiments on a series of widely used nonlinear control tasks show that our method can synthesize large safe weight sets with probability measure as high as 95\% even for a large-scale system of dimension 7.

Safety Verification of Nonlinear Systems with Bayesian Neural Network Controllers

Achieving gender equality is an important pillar for humankind’s sustainable future. Pioneering data-driven gender bias research is based on large-scale public records such as scientific papers, patents, and company registrations, covering female researchers, inventors and entrepreneurs, and so on. Since gender information is often missing in relevant datasets, studies rely on tools to infer genders from names. However, available open-sourced Chinese gender-guessing tools are not yet suitable for scientific purposes, which may be partially responsible for female Chinese being underrepresented in mainstream gender bias research and affect their universality. Specifically, these tools focus on character-level information while overlooking the fact that the combinations of Chinese characters in multi-character names, as well as the components and pronunciations of characters, convey important messages. As a first effort, we design a Chinese Heterogeneous Graph Attention (CHGAT) model to capture the heterogeneity in component relationships and incorporate the pronunciations of characters. Our model largely surpasses current tools and also outperforms the state-of-the-art algorithm. Last but not least, the most popular Chinese name-gender dataset is single-character based with far less female coverage from an unreliable source, naturally hindering relevant studies. We open-source a more balanced multi-character dataset from an official source together with our code, hoping to help future research promoting gender equality.

For the Underrepresented in Gender Bias Research: Chinese Name Gender Prediction with Heterogeneous Graph Attention Network

Trustworthy Residual Vehicle Value Prediction for Auto Finance

Deep learning models have achieved tremendous successes in accurate predictions for computer vision, natural language processing and speech recognition applications. However, to succeed in high-risk and safety-critical domains such as healthcare and finance, these deep learning models need to be made reliable and trustworthy. Specifically, they need to be robust and adaptive to real-world environments which can be drastically different from the training settings. In this talk, I will advocate for Bayesian principles to achieve the goal of building robust and adaptive deep learning models. I will introduce a suite of uncertainty quantification methods for Bayesian deep learning, and demonstrate applications enabled by accurate uncertainty estimates, e.g., robust prediction, continual learning and repairing model failures. I will conclude by discussing the research challenges for robust and adaptive deep learning models.

Robust and Adaptive Deep Learning via Bayesian Principles

In recent years, disinformation and fake news have become a global phenomenon, particularly so during the pandemic, COVID-19. The wide dissemination of fake news can have detrimental societal effects on individuals and the society. Despite the recent progress in detecting fake news, it remains a challenging task due to its complexity, diversity, multi-modality, speed, and costs of fact-checking or annotation. In this talk, we look at some challenges and lessons learned when exploring effective and explainable detecting strategies of fake news, and discuss challenges in fake news research and the need for interdisciplinary research.

Combating Disinformation on Social Media and Its Challenges: A Computational Perspective

In the talk, I shall describe my lab’s recent advances in AI, applied machine learning, and data mining to combat malicious actors (sockpuppets, ban evaders, etc.) and dangerous content (misinformation, hate, etc.) on web and social media platforms. My vision is to create a trustworthy online ecosystem for everyone and create the next generation of socially-aware methods that promote health, equity, and safety. Broadly, in my research, I have created novel graph, content (NLP, multimodality), and adversarial machine learning methods leveraging terabytes of data to detect, predict, and mitigate online threats. My interdisciplinary research innovates socio-technical solutions that I achieve by amalgamating computer science with social science theories. I am also passionate about putting my re- search into practice – my lab’s models have been deployed on Flipkart, influenced Twitter’s Birdwatch, and now being deployed on Wikipedia. My research has started a paradigm shift from the current slow and reactive approach against online harms to agile, proactive, and whole-of-society solutions. My talk will outline four thrusts of my research:
(1) Detection of harmful content and malicious actors across platforms, languages, and modalities
(2) Robustifying detection models against adversarial actors by predicting future malicious activities
(3) Attributing the impact of harmful content and the role of recommender systems
(4) Mitigation techniques to counter misinformation by professionals and non-expert crowds

Premium content

Unifying Vision-Language Representation Space with Single-tower Transformer

Downloads

Next from AAAI 2023

Modeling Human Trust and Reliance in AI-assisted Decision Making: A Markovian Approach

Similar lecture

Geometric Inductive Biases for Identifiable Unsupervised Learning of Disentangled Representations