United States

Patch deformation-based methods have recently exhibited substantial effectiveness in multi-view stereo, due to the incorporation of deformable and expandable perception to reconstruct textureless areas. 
However, such approaches typically focus on exploring correlative reliable pixels to alleviate match ambiguity during patch deformation, but ignore the deformation instability caused by mistaken edge-skipping and visibility occlusion, leading to potential estimation deviation. 
To remedy above issues, we propose DVP-MVS, which innovatively synergizes depth-edge aligned and cross-view priors for robust and visibility-aware patch deformation.
Specifically, to avoid unexpected edge-skipping, we first utilize Depth Anything V2 followed by the Roberts operator to initialize coarse depth and edge maps respectively, both of which are further aligned through an erosion-dilation strategy to generate fine-grained homogeneous boundaries for guiding patch deformation.
In addition, we reform view selection weights as visibility maps and restore visible areas by cross-view depth reprojection, then regard them as cross-view prior to facilitate visibility-aware patch deformation. 
Finally, we improve propagation and refinement with multi-view geometry consistency by introducing aggregated visible hemispherical normals based on view selection and local projection depth differences based on epipolar lines, respectively.
Extensive evaluations on ETH3D and Tanks &amp; Temples benchmarks demonstrate that our method can achieve state-of-the-art performance with excellent robustness and generalization.

AAAI 2025

DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View Stereo

3d computer vision

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Patch deformation-based methods have recently exhibited substantial effectiveness in multi-view stereo, due to the incorporation of deformable and expandable perception to reconstruct textureless areas. 
However, such approaches typically focus on exploring correlative reliable pixels to alleviate match ambiguity during patch deformation, but ignore the deformation instability caused by mistaken edge-skipping and visibility occlusion, leading to potential estimation deviation. 
To remedy above issues, we propose DVP-MVS, which innovatively synergizes depth-edge aligned and cross-view priors for robust and visibility-aware patch deformation.
Specifically, to avoid unexpected edge-skipping, we first utilize Depth Anything V2 followed by the Roberts operator to initialize coarse depth and edge maps respectively, both of which are further aligned through an erosion-dilation strategy to generate fine-grained homogeneous boundaries for guiding patch deformation.
In addition, we reform view selection weights as visibility maps and restore visible areas by cross-view depth reprojection, then regard them as cross-view prior to facilitate visibility-aware patch deformation. 
Finally, we improve propagation and refinement with multi-view geometry consistency by introducing aggregated visible hemispherical normals based on view selection and local projection depth differences based on epipolar lines, respectively.
Extensive evaluations on ETH3D and Tanks & Temples benchmarks demonstrate that our method can achieve state-of-the-art performance with excellent robustness and generalization.

poster

Few-shot Continual Relation Extraction is a crucial challenge for enabling AI systems to identify and adapt to evolving relationships in dynamic real-world domains. Traditional memory-based approaches often overfit to limited samples, failing to reinforce old knowledge, with the scarcity of data in few-shot scenarios further exacerbating these issues by hindering effective data augmentation in the latent space. In this paper, we propose a novel retrieval-based solution, starting with a large language model to generate descriptions for each relation. From these descriptions, we introduce a bi-encoder retrieval training paradigm to enrich both sample and class representation learning. Leveraging these enhanced representations, we design a retrieval-based prediction method where each sample "retrieves" the best fitting relation via a reciprocal rank fusion score that integrates both relation description vectors and class prototypes. Extensive experiments on multiple datasets demonstrate that our method significantly advances the state-of-the-art by maintaining robust performance across sequential tasks, effectively addressing catastrophic forgetting.

Few-Shot, No Problem: Descriptive Continual Relation Extraction

Given an input video of a person and a new garment, the goal of this paper is to synthesize a new video of that person wearing the specified clothing item while maintaining spatio-temporal consistency. Although significant progress has been made in image-based virtual try-on, extending this success to videos often leads to inconsistencies across frames. Several approaches have tried to address temporal consistency in video  virtual try-on, but they encounter major challenges when dealing with long video sequences, including temporal incoherence and high computational costs. To tackle these issues, we reconceptualize video virtual try-on as a conditional video inpainting task, using garments as input conditions. In specific, our approach extends image diffusion models with temporal modules to enhance temporal awareness. To overcome high computational cost on long videos, we introduce ShiftCaching, a training-free technique that ensures temporal coherence while reducing redundant computations. Additionally, we present a new video try-on dataset, the TikTokDress dataset, which includes more complex backgrounds, challenging movements, and higher resolution than existing public datasets. Extensive experiments demonstrate that our approach significantly outperforms existing baselines in both the qualitative and quantitative aspects. Code and dataset will be made available upon paper acceptance.

SwiftTry: Fast and Consistent Video Virtual Try-On with Diffusion Models

Adversarial training is one of the most effective approaches against adversarial attacks.
However, adversarial training has primarily been studied in scenarios where data for all classes is provided, with limited research conducted in the context of incremental learning where knowledge is introduced sequentially.
In this study, we investigate Adversarially Robust Class Incremental Learning (ARCIL), which deals with adversarial robustness in incremental learning.
We first explore a series of baselines that integrate incremental learning with existing adversarial training methods, finding that they lead to conflicts between acquiring new knowledge and retaining past knowledge.
Furthermore, we discover that training new knowledge causes the disappearance of a key characteristic in robust models: a flat loss landscape in input space.
To address such issues, we propose a novel and robust baseline for ARCIL, named \textbf{FL}atness-preserving \textbf{A}dversarial \textbf{I}ncremental learning for \textbf{R}obustness (\textbf{FLAIR}).
Experimental results demonstrate that FLAIR significantly outperforms other baselines.
To the best of our knowledge, we are the first to comprehensively investigate the baselines, challenges, and solutions for ARCIL, which we believe represents a significant advance toward achieving real-world robustness.

Enhancing Robustness in Incremental Learning with Adversarial Training

Solving systems of ordinary differential equations (ODEs) is essential when it comes to understanding the behavior of dynamical systems. Yet, automated solving remains challenging, in particular for nonlinear systems.  Computer algebra systems (CASs) provide support for solving ODEs by first simplifying them, in particular through the use of Lie point symmetries. Finding these symmetries is, however, itself a difficult problem for CASs. Recent works in symbolic regression have shown promising results for recovering symbolic expressions from data. Here, we adapt search-based symbolic regression to the task of finding generators of Lie point symmetries. 
With this approach, we can find symmetries of ODEs that existing CASs cannot find.

Discovering Symmetries of ODEs by Symbolic Regression

Solutions of symbolic regression problems are expressions that are composed of input variables and operators from a finite set of function symbols. One measure for evaluating symbolic regression algorithms is their ability to recover formulae, up to symbolic equivalence, from finite samples. Not unexpectedly, the recovery problem becomes harder when the formula gets more complex, that is, when the number of variables and operators gets larger. Variables in naturally occurring symbolic formulas often appear only in fixed combinations. This can be exploited in symbolic regression by substituting one new variable for the combination, effectively reducing the number of variables. However, finding valid substitutions is challenging. Here, we address this challenge by searching over the expression space of small substitutions and testing for validity. The validity test is reduced to a test of functional dependence. The resulting iterative dimension reduction procedure can be used with any symbolic regression approach. We show that it reliably identifies valid substitutions and significantly boosts the performance of different types of state-of-the-art symbolic regression algorithms.

Dimension Reduction for Symbolic Regression

The independence assumption is a useful tool to increase the tractability of one's modelling framework.  However, this assumption does not match reality; failing to take dependencies into account can cause models to fail dramatically.  The field of multi-axis graphical modelling (also called multi-way modelling, Kronecker-separable modelling) has seen growth over the past decade, but these models require that the data have zero mean.  In the multi-axis case, inference is typically done in the single sample scenario, making mean inference impossible.

In this paper, we demonstrate how the zero-mean assumption can cause egregious modelling errors, as well as propose a relaxation to the zero-mean assumption that allows the avoidance of such errors.  Specifically, we propose the "Kronecker-sum-structured mean" assumption, which leads to models with nonconvex-but-unimodal log-likelihoods that can be solved efficiently with coordinate descent.

Gaussian Graphical Modelling Without Independence Assumptions for Uncentered Data

In recent years, there has been a growing demand to stylize a given 3D scene to align with the artistic style of reference images for creative purposes. While 3D Gaussian Splatting (GS) has emerged as a promising and efficient method for realistic 3D scene modeling, there remains a challenge in adapting it to stylize 3D GS to match with multiple styles through automatic local style transfer or manual designation, while maintaining memory efficiency for stylization training. In this paper, we introduce a novel 3D GS stylization solution termed Multi-StyleGS to tackle these challenges. In particular, we employ a bipartite matching mechanism to automatically identify correspondences between the style images and the local regions of the rendered images. To facilitate local style transfer, we introduce a novel semantic style loss function that employs a segmentation network to apply distinct styles to various objects of the scene and propose a local-global feature matching to enhance the multi-view consistency. Furthermore, this technique can achieve memory-efficient training, more texture details and better color match. To better assign a robust semantic label to each Gaussian, we propose several techniques to regularize the segmentation network. As demonstrated by our comprehensive experiments, our approach outperforms existing ones in producing plausible stylization results and offering flexible editing.

Multi-StyleGS: Stylized Gaussian Splatting with Multiple Styles

Event cameras encode visual information by generating asynchronous and sparse event streams, which hold great potential for low latency and low power consumption. Despite many successful implementations of event camera-based applications, most of them accumulate the events into frames and then utilize conventional frame-based computer vision algorithms. These frame-based methods, though typically effective, diminish the inherent advantages of the event camera's low latency and low power consumption. To solve the above problems, we propose ASGCN, which efficiently processes data on an event-by-event basis and dynamically evolves into a corresponding dynamic representation, enabling low latency and high sparsity of data representation. The sparsity computation is further improved by introducing brain-inspired spiking neural networks, resulting in low power consumption for ASGCN. Extensive and diverse experiments demonstrate the energy efficiency and low latency advantages of our processing pipeline. Especially on real-world event camera datasets, our pipeline consumes more than 10,000 times less energy and achieves similar performance compared to current frame-based methods.

Leveraging Asynchronous Spiking Neural Networks for Ultra Efficient Event-Based Visual Processing

Video scene detection involves assessing whether each shot and its surroundings belong to the same scene.
Achieving this requires meticulously correlating multi-modal cues, $\textit{e.g.}$, visual entity and place modalities, among shots and comparing semantic changes around each shot.
However, most methods treat multi-modal semantics equally and do not examine contextual differences between the two sides of a shot, leading to sub-optimal detection performance.
In this paper, we propose the $\textbf{M}$odality-$\textbf{A}$ware $\textbf{S}$hot $\textbf{R}$elating and $\textbf{C}$omparing approach (MASRC), which enables relating shots per their own characteristics of visual entity and place modalities, as well as comparing multi-shots similarities to have scene changes explicitly encoded. Specifically, 
to fully harness the potential of visual entity and place modalities in modeling shot relations, we mine long-term shot correlations from entity semantics while simultaneously revealing short-term shot correlations from place semantics.
In this way, we can learn distinctive shot features that consolidate coherence within scenes and amplify distinguishability across scenes.
Once equipped with distinctive shot features, we further encode the relations between preceding and succeeding shots of each target shot by similarity convolution, aiding in the identification of scene ending shots.
We validate the broad applicability of the proposed components in MASRC. Extensive experimental results on public benchmark datasets demonstrate that the proposed MASRC significantly advances video scene detection. The code will be released.

Modality-Aware Shot Relating and Comparing for Video Scene Detection

Partially relevant video retrieval (PRVR) is a task that performs video retrieval where only a specific segment of the video is relevant to a text query. Previously, typical approaches were to enhance the video clip modeling to prepare for queries describing diverse moments. In this paper, we tackle the underexplored problem of ambiguous relationships between text-video pairs. Typical training processes of PRVR assume a one-to-one relationship where each text query is relevant to only one video. However, we point out the inherent ambiguity between text and video content based on their conceptual scope and propose a framework that incorporates this ambiguity into the model learning process. To achieve this, we utilize online uncertainty and similarity measures during training to detect and exclude ambiguity in text-video representation learning. Specifically, we grant the model freedom to handle ambiguous relationships instead of forcing them to be negative. Additionally, we delve into fine-grained relationships within the video instance. Whereas only the relationships with a video as the base unit were considered previously to provide training guidance, we explore the various contexts within the same untrimmed video with our framework. Lastly, we employ the co-teaching framework to mitigate the impact of ambiguous labels and confirmation bias during ambiguity predictions within a single network. With all components combined, our proposed method demonstrates its effectiveness in PRVR.

Downloads

Next from AAAI 2025

Few-Shot, No Problem: Descriptive Continual Relation Extraction

Similar lecture

MSP-MVS: Multi-Granularity Segmentation Prior Guided Multi-View Stereo