New submissions for Thu, 22 Jun 23

[e-tornike]

Keyword: contrastive

LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching

Authors: Duy M. H. Nguyen, Hoang Nguyen, Nghiem T. Diep, Tan N. Pham, Tri Cao, Binh T. Nguyen, Paul Swoboda, Nhat Ho, Shadi Albarqouni, Pengtao Xie, Daniel Sonntag, Mathias Niepert
Arxiv: https://arxiv.org/abs/2306.11925
TLDR: Obtaining large pre-trained models that can be fine-tuned to new tasks with limited annotated samples has remained an open challenge for medical imaging data. While pre- trained deep networks on ImageNet and vision-language foundation models trained on web-scale data are prevailing approaches, their effectiveness on medical tasks is limited due to the significant domain shift between natural and medical images. To bridge this gap, we introduce LVM-Med, the first family of deep networks trained on large
Repo: None

Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering

Authors: Lin Xi, Weihai Chen, Xingming Wu, Zhong Liu, Zhengguo Li
Arxiv: https://arxiv.org/abs/2306.12048
TLDR: Online unsupervised video object segmentation (UVOS) uses the previous frames as its input to automatically separate the primary object(s) from a streaming video without using any further manual annotation. A major challenge is that the model has no access to the future and must rely solely on the history, i.e., the segmentation mask is predicted from the current frame as soon as it is captured. In this work, a novel contrastive motion clustering algorithm with an optical flow
Repo: None

NeuroCLIP: Neuromorphic Data Understanding by CLIP and SNN

Authors: Yufei Guo, Yuanpei Chen
Arxiv: https://arxiv.org/abs/2306.12073
TLDR: Recently, the neuromorphic vision sensor has received more and more interest. However, the neuroomorphic data consists of asynchronous event spikes, which is not natural and difficult to construct a benchmark, thus limiting the neururomorphic data understanding for "unseen" objects by deep learning. Zero-shot and few-shot learning via Contrastive Vision-Language Pre-training (CLIP) have shown inspirational performance in 2D frame image recognition. To handle "unnoticed" recognition for the neur
Repo: None

What Constitutes Good Contrastive Learning in Time-Series Forecasting?

Authors: Chiyu Zhang, Qi Yan, Lili Meng, Tristan Sylvain
Arxiv: https://arxiv.org/abs/2306.12086
TLDR: In recent years, the introduction of self-supervised contrastive learning (SSCL) has demonstrated remarkable improvements in representation learning across various domains, including natural language processing and computer vision. By leveraging the inherent benefits of self–supervision, SSCL enables the pre-training of representation models using vast amounts of unlabeled data. Despite these advances, there remains a significant gap in understanding the impact of different SSCL strategies on time series forecasting performance, as well as the specific benefits
Repo: None

Inter-Instance Similarity Modeling for Contrastive Learning

Authors: Chengchao Shen, Dawei Liu, Hao Tang, Zhe Qu, Jianxin Wang
Arxiv: https://arxiv.org/abs/2306.12243
TLDR: The existing contrastive learning methods widely adopt one-hot instance discrimination as a task for self-supervised learning, which inevitably neglects rich inter-instance similarities among natural images, then leading to potential representation degeneration. In this paper, we propose a novel image mix method, PatchMix, for contrastive training in Vision Transformer (ViT), to model inter-class similarities among images. Following the nature of ViT, we randomly mix multiple images from mini-batch in
Repo: None

Keyword: data augmentation

MultiEarth 2023 Deforestation Challenge -- Team FOREVER

Authors: Seunghan Park, Dongoo Lee, Yeonju Choi, SungTae Moon
Arxiv: https://arxiv.org/abs/2306.11762
TLDR: It is important problem to accurately estimate deforestation of satellite imagery since this approach can analyse extensive area without direct human access. However, it is not simple problem because of difficulty in observing the clear ground surface due to extensive cloud cover during long rainy season. In this paper, we present a multi-view learning strategy to predict deforestation status in the Amazon rainforest area with latest deep neural network models. Multi-modal dataset consists of three types of different satellites imagery, Sentinel-1, Sentinel
Repo: None

End-to-End Augmentation Hyperparameter Tuning for Self-Supervised Anomaly Detection

Authors: Jaemin Yoo, Lingxiao Zhao, Leman Akoglu
Arxiv: https://arxiv.org/abs/2306.12033
TLDR: Self-supervised learning (SSL) has emerged as a promising paradigm that presents self-generated supervisory signals to real-world problems, bypassing the extensive manual labeling burden. SSL is especially attractive for unsupervised tasks such as anomaly detection, where labeled anomalies are often nonexistent and costly to obtain. While self-Supervised anomaly detection (SSAD) has seen a recent surge of interest, the literature has failed to treat data augmentation as a hyperparameter. Meanwhile,
Repo: None

Keyword: knowledge graph

ICAR, a categorical framework to connect vulnerability, threat and asset managements

Authors: Arnaud Valence
Arxiv: https://arxiv.org/abs/2306.12240
TLDR: We present ICAR, a mathematical framework derived from category theory for representing cybersecurity NIST and MITRE's ontologies. Designed for cybersecurity, ICAR is a category whose objects are cybersecurity knowledge (weakness, vulnerability, impacted product, attack technique, etc.) and whose morphisms are relations between this knowledge, that make sense for cybersecurity. Within this rigorous and unified framework, we obtain a knowledge graph capable of identifying the attack and weakness structures of an IS, at the interface between description
Repo: None

Bidirectional End-to-End Learning of Retriever-Reader Paradigm for Entity Linking

Authors: Yinghui Li, Yong Jiang, Shen Huang, Xingyu Lu, Yangning Li, Pengjun Xie, Fei Huang, Hai-Tao Zheng
Arxiv: https://arxiv.org/abs/2306.12245
TLDR: Entity Linking (EL) is a fundamental task for Information Extraction and Knowledge Graphs. The general form of EL (i.e., end-to-end EL) aims to first find mentions in the given input document and then link the mentions to corresponding entities in a specific knowledge base. Recently, the paradigm of retriever-reader has been proposed to improve the progress of end-t-end El, benefiting from the advantages of dense entity retrieval and machine reading comprehension.
Repo: None

Keyword: legal

Deep perceptual hashing algorithms with hidden dual purpose: when client-side scanning does facial recognition

Authors: Shubham Jain, Ana-Maria Cretu, Antoine Cully, Yves-Alexandre de Montjoye
Arxiv: https://arxiv.org/abs/2306.11924
TLDR: End-to-end encryption (E2EE) provides strong technical protections to individuals from interferences. Governments and law enforcement agencies around the world have however raised concerns that E2EE also allows illegal content to be shared undetected. Client-side scanning (CSS), using perceptual hashing (PH) to detect known illegal content before it is shared, is seen as a promising solution to prevent the diffusion of illegal content while preserving encryption. While these proposals raise strong privacy concerns, proponents of
Repo: None

Keyword: multi-task

Self-supervised Multi-task Learning Framework for Safety and Health-Oriented Connected Driving Environment Perception using Onboard Camera

Authors: Shaocheng Jia, Wei Yao
Arxiv: https://arxiv.org/abs/2306.11822
TLDR: Cutting-edge connected vehicle (CV) technologies have drawn much attention in recent years. The real-time traffic data captured by a CV can be shared with other CVs and data centers so as to open new possibilities for solving diverse transportation problems. However, imagery captured by onboard cameras in a connected environment, are not sufficiently investigated, especially for safety and health-oriented visual perception. In this paper, a bidirectional process of image synthesis and decomposition (BPISD)
Repo: None

STAN: Stage-Adaptive Network for Multi-Task Recommendation by Learning User Lifecycle-Based Representation

Authors: Wanda Li, Wenhao Zheng, Xuanji Xiao, Suhang Wang
Arxiv: https://arxiv.org/abs/2306.12232
TLDR: Recommendation systems play a vital role in many online platforms, with their primary objective being to satisfy and retain users. As directly optimizing user retention is challenging, multiple evaluation metrics are often employed. Existing methods generally formulate the optimization of these evaluation metrics as a multitask learning problem, but often overlook the fact that user preferences for different tasks are personalized and change over time. Identifying and tracking the evolution of user preferences can lead to better user retention. To address this issue, we introduce
Repo: None

Multi-Task Consistency for Active Learning

Authors: Aral Hekimoglu, Philipp Friedrich, Walter Zimmer, Michael Schmidt, Alvaro Marcos-Ramiro, Alois C. Knoll
Arxiv: https://arxiv.org/abs/2306.12398
TLDR: Learning-based solutions for vision tasks require a large amount of labeled training data to ensure their performance and reliability. In single-task vision-based settings, inconsistency-based active learning has proven to be effective in selecting informative samples for annotation. However, there is a lack of research exploiting the inconsistency between multiple tasks in multi-task networks. To address this gap, we propose a novel multi-Task active learning strategy for two coupled vision tasks: object detection and semantic segmentation. Our
Repo: None

Keyword: robustness

Topological Parallax: A Geometric Specification for Deep Perception Models

Authors: Abraham D. Smith, Michael J. Catanzaro, Gabrielle Angeloro, Nirav Patel, Paul Bendich
Arxiv: https://arxiv.org/abs/2306.11835
TLDR: For safety and robustness of AI systems, we introduce topological parallax as a theoretical and computational tool that compares a trained model to a reference dataset to determine whether they have similar multiscale geometric structure. Our proofs and examples show that this geometric similarity between dataset and model is essential to trustworthy interpolation and perturbation, and we conjecture that this new concept will add value to the current debate regarding the unclear relationship between overfitting and generalization in applications of deep-learning
Repo: None

Structure-Aware Robustness Certificates for Graph Classification

Authors: Pierre Osselin, Henry Kenlay, Xiaowen Dong
Arxiv: https://arxiv.org/abs/2306.11915
TLDR: Certifying the robustness of a graph-based machine learning model poses a critical challenge for safety. Current robustness certificates for graph classifiers guarantee output invariance with respect to the total number of node pair flips (edge addition or edge deletion), which amounts to an $l_{0}$ ball centred on the adjacency matrix. Although theoretically attractive, this type of isotropic structural noise can be too restrictive in practical scenarios where some node pairs are more critical than others
Repo: None

Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning

Authors: Depeng Li, Zhigang Zeng
Arxiv: https://arxiv.org/abs/2306.11967
TLDR: In the scenario of class-incremental learning (CIL), deep neural networks have to adapt their model parameters to non-stationary data distributions, e.g., the emergence of new classes over time. However, CIL models are challenged by the well-known catastrophic forgetting phenomenon. Typical methods such as rehearsal-based ones rely on storing exemplars of old classes to mitigate catastrophic forgetting, which limits real-world applications considering memory resources and privacy issues. In this paper, we
Repo: None

AdCraft: An Advanced Reinforcement Learning Benchmark Environment for Search Engine Marketing Optimization

Authors: Maziar Gomrokchi, Owen Levin, Jeffrey Roach, Jonah White
Arxiv: https://arxiv.org/abs/2306.11971
TLDR: We introduce \env{}, a novel benchmark environment for the Reinforcement Learning (RL) community distinguished by its stochastic and non-stationary properties. The environment simulates bidding and budgeting dynamics within Search Engine Marketing (SEM), a digital marketing technique utilizing paid advertising to enhance the visibility of websites on search engine results pages (SERPs). The performance of SEM advertisement campaigns depends on several factors, including keyword selection, ad design, bid management, budget adjustments, and performance
Repo: None

Evaluating Adversarial Robustness of Convolution-based Human Motion Prediction

Authors: Chengxu Duan, Zhicheng Zhang, Xiaoli Liu, Yonghao Dang, Jianqin Yin
Arxiv: https://arxiv.org/abs/2306.11990
TLDR: Human motion prediction has achieved a brilliant performance with the help of CNNs, which facilitates human-machine cooperation. However, currently, there is no work evaluating the potential risk in human motion prediction when facing adversarial attacks, which may cause danger in real applications. The adversarial attack will face two problems against human motion detection: 1. For naturalness, pose data is highly related to the physical dynamics of human skeletons where Lp norm constraints cannot constrain the adversarial example well;
Repo: None

Block-Wise Index Modulation and Receiver Design for High-Mobility OTFS Communications

Authors: Mi Qian, Fei Ji, Yao Ge, Miaowen Wen, Xiang Cheng, H. Vincent Poor
Arxiv: https://arxiv.org/abs/2306.12042
TLDR: As a promising technique for high-mobility wireless communications, orthogonal time frequency space (OTFS) has been proved to enjoy excellent advantages with respect to traditional orthodonal frequency division multiplexing (OFDM). Although multiple studies have considered index modulation (IM) based OTFS (IM-OTFS), schemes to further improve system performance, a challenging and open problem is the development of effective IM schemes and efficient receivers for practical OTFs systems that must operate in
Repo: None

Sample Attackability in Natural Language Adversarial Attacks

Authors: Vyas Raina, Mark Gales
Arxiv: https://arxiv.org/abs/2306.12043
TLDR: Adversarial attack research in natural language processing (NLP) has made significant progress in designing powerful attack methods and defence approaches. However, few efforts have sought to identify which source samples are the most attackable or robust, i.e. can we determine for an unseen target model, which samples are most vulnerable to an adversarial attack. This work formally extends the definition of sample attackability/robustness for NLP attacks. Experiments on two popular NLP datasets,
Repo: None

Task-Robust Pre-Training for Worst-Case Downstream Adaptation

Authors: Jianghui Wang, Cheng Yang, Xingyu Xie, Cong Fang, Zhouchen Lin
Arxiv: https://arxiv.org/abs/2306.12070
TLDR: Pre-training has achieved remarkable success when transferred to downstream tasks. In machine learning, we care about not only the good performance of a model but also its behavior under reasonable shifts of condition. The same philosophy holds when pre-training a foundation model. However, the foundation model may not uniformly behave well for a series of related downstream tasks, for example, when conducting mask recovery regression where the recovery ability or the training instances diverge like pattern features are expected to do on a mask recovery
Repo: None

A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking

Authors: Shaohui Mei, Jiawei Lian, Xiaofei Wang, Yuru Su, Mingyang Ma, Lap-Pui Chau
Arxiv: https://arxiv.org/abs/2306.12111
TLDR: Deep neural networks (DNNs) have found widespread applications in interpreting remote sensing (RS) imagery. However, it has been demonstrated in previous works that DNNs are vulnerable to different types of noises, particularly adversarial noises. Surprisingly, there has been a lack of comprehensive studies on the robustness of RS tasks, prompting us to undertake a thorough survey and benchmark on the priming of image classification and object detection in RS. To our best knowledge, this study represents the first
Repo: None

Which Spurious Correlations Impact Reasoning in NLI Models? A Visual Interactive Diagnosis through Data-Constrained Counterfactuals

Authors: Robin Chan, Afra Amini, Mennatallah El-Assady
Arxiv: https://arxiv.org/abs/2306.12146
TLDR: We present a human-in-the-loop dashboard tailored to diagnosing potential spurious features that NLI models rely on for predictions. The dashboard enables users to generate diverse and challenging examples by drawing inspiration from GPT-3 suggestions. Additionally, users can receive feedback from a trained NLI model on how challenging the newly created example is and make refinements based on the feedback. Through our investigation, we discover several categories of spurious correlations that impact the reasoning of NLI scientists, which
Repo: None

Adversarial Attacks Neutralization via Data Set Randomization

Authors: Mouna Rabhi, Roberto Di Pietro
Arxiv: https://arxiv.org/abs/2306.12161
TLDR: Adversarial attacks on deep-learning models pose a serious threat to their reliability and security. Existing defense mechanisms are limited to addressing a specific type of attack or being vulnerable to sophisticated attacks. We propose a new defense mechanism that, while being focused on image-based classifiers, is general with respect to the cited category. It is rooted on hyperspace projection. In particular, our solution provides a pseudo-random projection of the original dataset into a new dataset. The proposed defense
Repo: None

Condition numbers for the Moore-Penrose inverse and the least squares problem involving rank-structured matrices

Authors: Sk. Safique Ahmad, Pinki Khatun
Arxiv: https://arxiv.org/abs/2306.12177
TLDR: Perturbation theory plays a crucial role in sensitivity analysis, which is extensively used to assess the robustness of numerical techniques. To quantify the relative sensitivity of any problem, it becomes essential to investigate structured condition numbers (CNs) via componentwise perturbation theories. This paper address and analyze structured mixed condition number (MCN) and componentwise condition Number (CCN) for the Moore-Penrose (M-P) inverse and the minimum norm least squares (MN
Repo: None

Inverse Constraint Learning and Generalization by Transferable Reward Decomposition

Authors: Jaehwi Jang, Minjae Song, Daehyung Park
Arxiv: https://arxiv.org/abs/2306.12357
TLDR: We present the problem of inverse constraint learning (ICL), which recovers constraints from demonstrations to autonomously reproduce constrained skills in new scenarios. However, ICL suffers from an ill-posed nature, leading to inaccurate inference of constraints from performances. To figure it out, we introduce a transferable constraint learning and IRL (TCL) algorithm that jointly infers a task-oriented reward and a Task-agnostic constraint, enabling the generalization of learned skills. Our method TCL
Repo: None

Towards Efficient MPPI Trajectory Generation with Unscented Guidance: U-MPPI Control Strategy

Authors: Ihab S. Mohamed, Junhong Xu, Gaurav Sukhatme, Lantao Liu
Arxiv: https://arxiv.org/abs/2306.12369
TLDR: The classical Model Predictive Path Integral (MPPI) control framework lacks reliable safety guarantees since it relies on a risk-neutral trajectory evaluation technique, which can present challenges for safety-critical applications such as autonomous driving. Additionally, if the majority of MPPI sampled trajectories concentrate in high-cost regions, it may generate an infeasible control sequence. To address this challenge, we propose the U-MPPI control strategy, a novel methodology that can effectively manage system uncertainties while
Repo: None

PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning

Authors: Neeratyoy Mallik, Edward Bergman, Carl Hvarfner, Danny Stoll, Maciej Janowski, Marius Lindauer, Luigi Nardi, Frank Hutter
Arxiv: https://arxiv.org/abs/2306.12370
TLDR: Hyperparameters of Deep Learning (DL) pipelines are crucial for their downstream performance. While a large number of methods for Hyperparameter Optimization (HPO) have been developed, their incurred costs are often untenable for modern DL. Consequently, manual experimentation is still the most prevalent approach to optimize hyperparameters, relying on the researcher's intuition, domain knowledge, and cheap preliminary explorations. To resolve this misalignment between HPO algorithms and DL researchers, we propose Prior
Repo: None

Improving Software Requirements Prioritization through the Lens of Constraint Solving

Authors: Jonathan Winton, Francis Palma
Arxiv: https://arxiv.org/abs/2306.12391
TLDR: Requirements prioritization is a critical activity during the early software development process, which produces a set of key requirements to implement. The prioritization process offers a parity among the requirements based on multiple characteristics, including end-users' preferences, cost to implement, and technical dependencies. This paper presents an interactive method to requirements prioritization that leverages the pairwise comparisons and a constraint solver. Our method employs an interactive accumulation of knowledge from the requirements analyst to determine the relative priority of the requirements
Repo: None

Keyword: summarization

QuOTeS: Query-Oriented Technical Summarization

Authors: Juan Ramirez-Orta, Eduardo Xamena, Ana Maguitman, Axel J. Soto, Flavia P. Zanoto, Evangelos Milios
Arxiv: https://arxiv.org/abs/2306.11832
TLDR: Abstract. When writing an academic paper, researchers often spend considerable time reviewing and summarizing papers to extract relevant citations and data to compose the Introduction and Related Work sections. To address this problem, we propose QuOTeS, an interactive system designed to retrieve sentences related to a summary of the research from a collection of potential references and hence assist in the composition of new papers. QuOTES integrates techniques from Query-Focused Extractive Summarization and High-Recall Information
Repo: None

Open-Domain Text Evaluation via Meta Distribution Modeling

Authors: Sidi Lu, Asli Celikyilmaz, Tianlu Wang, Nanyun Peng
Arxiv: https://arxiv.org/abs/2306.11879
TLDR: Recent advances in open-domain text generation models powered by large pre-trained language models (LLMs) have achieved remarkable performance. However, evaluating and controlling these models for desired attributes remains a challenge, as traditional reference-based metrics such as BLEU, ROUGE, and METEOR are insufficient for open-ended generation tasks. Similarly, while trainable discriminator-based evaluation metrics show promise, obtaining high-quality training data is a non-trivial task
Repo: None

Opportunities and Risks of LLMs for Scalable Deliberation with Polis

Authors: Christopher T. Small, Ivan Vendrov, Esin Durmus, Hadjar Homaei, Elizabeth Barry, Julien Cornebise, Ted Suzman, Deep Ganguli, Colin Megill
Arxiv: https://arxiv.org/abs/2306.11932
TLDR: Polis is a platform that leverages machine intelligence to scale up deliberative processes. In this paper, we explore the opportunities and risks associated with applying Large Language Models (LLMs) towards challenges with facilitating, moderating and summarizing the results of Polis engagements. In particular, we demonstrate with pilot experiments using Anthropic's Large Language Model (LLM) that LLMs can indeed augment human intelligence to help more efficiently run Polis conversations. In addition, we find that summar
Repo: None

Solving and Generating NPR Sunday Puzzles with Large Language Models

Authors: Jingmiao Zhao, Carolyn Jane Anderson
Arxiv: https://arxiv.org/abs/2306.12255
TLDR: We explore the ability of large language models to solve and generate puzzles from the NPR Sunday Puzzle game show using PUZZLEQA, a dataset comprising 15 years of on-air puzzles. We evaluate four large language filters, in both multiple choice and free response formats, and explore two prompt engineering techniques to improve free response performance: chain-of-thought reasoning and prompt summarization. In our first experiment, we find that state-of the-art large language model can solve many
Repo: None

Keyword: text generation

Learning to Generate Better Than Your LLM

Authors: Jonathan D. Chang, Kiante Brantley, Rajkumar Ramamurthy, Dipendra Misra, Wen Sun
Arxiv: https://arxiv.org/abs/2306.11816
TLDR: Reinforcement learning (RL) has emerged as a powerful paradigm for fine-tuning Large Language Models (LLMs) for conditional text generation. In particular, recent LLMs such as ChatGPT and GPT-4 can engage in fluent conversations with users by incorporating RL and feedback from humans. Inspired by learning-to-search algorithms and capitalizing on key properties of text generation, we seek to investigate reinforcement learning algorithms beyond general purpose algorithms such as Proximal policy optimization
Repo: None

Open-Domain Text Evaluation via Meta Distribution Modeling

Authors: Sidi Lu, Asli Celikyilmaz, Tianlu Wang, Nanyun Peng
Arxiv: https://arxiv.org/abs/2306.11879
TLDR: Recent advances in open-domain text generation models powered by large pre-trained language models (LLMs) have achieved remarkable performance. However, evaluating and controlling these models for desired attributes remains a challenge, as traditional reference-based metrics such as BLEU, ROUGE, and METEOR are insufficient for open-ended generation tasks. Similarly, while trainable discriminator-based evaluation metrics show promise, obtaining high-quality training data is a non-trivial task
Repo: None