Image inversion is a fundamental task in generative models, aiming to mapimages back to their latent representations to enable downstream applicationssuch as editing, restoration, and style transfer. This paper provides acomprehensive review of the latest advancements in image inversion techniques,focusing on two main paradigms: Generative Adversarial Network (GAN) inversionand diffusion model inversion. We categorize these techniques based on theiroptimization methods. For GAN inversion, we systematically classify existingmethods into encoder-based approaches, latent optimization approaches, andhybrid approaches, analyzing their theoretical foundations, technicalinnovations, and practical trade-offs. For diffusion model inversion, weexplore training-free strategies, fine-tuning methods, and the design ofadditional trainable modules, highlighting their unique advantages andlimitations. Additionally, we discuss several popular downstream applicationsand emerging applications beyond image tasks, identifying current challengesand future research directions. By synthesizing the latest developments, thispaper aims to provide researchers and practitioners with a valuable referenceresource, promoting further advancements in the field of image inversion. Wekeep track of the latest works at https://github.com/RyanChenYN/ImageInversion <<<

Denoising diffusion probabilistic models (DDPM) are powerful hierarchicallatent variable models with remarkable sample generation quality and trainingstability. These properties can be attributed to parameter sharing in thegenerative hierarchy, as well as a parameter-free diffusion-based inferenceprocedure. In this paper, we present Few-Shot Diffusion Models (FSDM), aframework for few-shot generation leveraging conditional DDPMs. FSDMs aretrained to adapt the generative process conditioned on a small set of imagesfrom a given class by aggregating image patch information using a set-basedVision Transformer (ViT). At test time, the model is able to generate samplesfrom previously unseen classes conditioned on as few as 5 samples from thatclass. We empirically show that FSDM can perform few-shot generation andtransfer to new datasets. We benchmark variants of our method on complex visiondatasets for few-shot learning and compare to unconditional and conditionalDDPM baselines. Additionally, we show how conditioning the model on patch-basedinput set information improves training convergence. <<<

Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, Daniel A. McFarland, James Y. Zou <<<

We present an approach for estimating the fraction of text in a large corpuswhich is likely to be substantially modified or produced by a large languagemodel (LLM). Our maximum likelihood model leverages expert-written andAI-generated reference texts to accurately and efficiently examine real-worldLLM-use at the corpus level. We apply this approach to a case study ofscientific peer review in AI conferences that took place after the release ofChatGPT: ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023. Our results suggestthat between 6.5% and 16.9% of text submitted as peer reviews to theseconferences could have been substantially modified by LLMs, i.e. beyondspell-checking or minor writing updates. The circumstances in which generatedtext occurs offer insight into user behavior: the estimated fraction ofLLM-generated text is higher in reviews which report lower confidence, weresubmitted close to the deadline, and from reviewers who are less likely torespond to author rebuttals. We also observe corpus-level trends in generatedtext which may be too subtle to detect at the individual level, and discuss theimplications of such trends on peer review. We call for futureinterdisciplinary work to examine how LLM use is changing our information andknowledge practices. <<<

Qiming Sun, Timothy C. Berkelbach, Nick S. Blunt, George H. Booth, Sheng Guo, Zhendong Li, Junzi Liu, James McClain, Elvira R. Sayfutyarova, Sandeep Sharma, Sebastian Wouters, Garnet Kin-Lic Chan <<<

PySCF is a general-purpose electronic structure platform designed from theground up to emphasize code simplicity, both to aid new method development, aswell as for flexibility in computational workflow. The package provides a widerange of tools to support simulations of finite size systems, extended systemswith periodic boundary conditions, low dimensional periodic systems, and customHamiltonians, using mean-field and post-mean-field methods with standardGaussian basis functions. To ensure easy of extensibility, PySCF uses thePython language to implement almost all its features, while computationallycritical paths are implemented with heavily optimized C routines. Using thiscombined Python/C implementation, the package is as efficient as the bestexisting C or Fortran based quantum chemistry programs. In this paper wedocument the capabilities and design philosophy of the current version of thePySCF package. <<<

Recent advancements in quantum computing have opened new avenues for tacklinglong-standing complex combinatorial optimization problems that are intractablefor classical computers. Predicting secondary structure of mRNA is one suchnotoriously difficult problem that can benefit from the ever-increasingmaturity of quantum computing technology. Accurate prediction of mRNA secondarystructure is critical in designing RNA-based therapeutics as it dictatesvarious steps of an mRNA life cycle, including transcription, translation, anddecay. The current generation of quantum computers have reached utility-scale,allowing us to explore relatively large problem sizes. In this paper, weexamine the feasibility of solving mRNA secondary structures on a quantumcomputer with sequence length up to 60 nucleotides representing problems in thequbit range of 10 to 80. We use Conditional Value at Risk (CVaR)-based VQEalgorithm to solve the optimization problems, originating from the mRNAstructure prediction problem, on the IBM Eagle and Heron quantum processors. Toour encouragement, even with ``minimal'' error mitigation and fixed-depthcircuits, our hardware runs yield accurate predictions of minimum free energy(MFE) structures that match the results of the classical solver CPLEX. Ourresults provide sufficient evidence for the viability of solving mRNA structureprediction problems on a quantum computer and motivate continued research inthis direction. <<<

Andrew W. Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Žídek, Alexander W. R. Nelson, Alex Bridgland, Hugo Penedones, Stig Petersen, Karen Simonyan, Steve Crossan, Pushmeet Kohli, David T. Jones, David Silver, Koray Kavukcuoglu, Demis Hassabis <<<

AbstractIn this article, we will introduce the basic concept and the quantum feature of a novel computing system, coherent Ising machines, and describe their theoretical and experimental performance. We start with the discussion how to construct such physical devices as the quantum analog of classical neuron and synapse, and end with the performance comparison against various classical neural networks implemented in CPU and supercomputers. <<<

The conformational landscape of proteins is crucial to understanding theirfunctionality in complex biological processes. Traditional physics-basedcomputational methods, such as molecular dynamics (MD) simulations, suffer fromrare event sampling and long equilibration time problems, hindering theirapplications in general protein systems. Recently, deep generative modelingtechniques, especially diffusion models, have been employed to generate novelprotein conformations. However, existing score-based diffusion methods cannotproperly incorporate important physical prior knowledge to guide the generationprocess, causing large deviations in the sampled protein conformations from theequilibrium distribution. In this paper, to overcome these limitations, wepropose a force-guided SE(3) diffusion model, ConfDiff, for proteinconformation generation. By incorporating a force-guided network with a mixtureof data-based score models, ConfDiff can can generate protein conformationswith rich diversity while preserving high fidelity. Experiments on a variety ofprotein conformation prediction tasks, including 12 fast-folding proteins andthe Bovine Pancreatic Trypsin Inhibitor (BPTI), demonstrate that our methodsurpasses the state-of-the-art method. <<<

There is a growing body of work that leverages features extracted viatopological data analysis to train machine learning models. While this field,sometimes known as topological machine learning (TML), has seen some notablesuccesses, an understanding of how the process of learning from topologicalfeatures differs from the process of learning from raw data is still limited.In this work, we begin to address one component of this larger issue by askingwhether a model trained with topological features learns internalrepresentations of data that are fundamentally different than those learned bya model trained with the original raw data. To quantify ``different'', weexploit two popular metrics that can be used to measure the similarity of thehidden representations of data within neural networks, neural stitching andcentered kernel alignment. From these we draw a range of conclusions about howtraining with topological features does and does not change the representationsthat a model learns. Perhaps unsurprisingly, we find that structurally, thehidden representations of models trained and evaluated on topological featuresdiffer substantially compared to those trained and evaluated on thecorresponding raw data. On the other hand, our experiments show that in somecases, these representations can be reconciled (at least to the degree requiredto solve the corresponding task) using a simple affine transformation. Weconjecture that this means that neural networks trained on raw data may extractsome limited topological features in the process of making predictions. <<<

Topological Data Analysis is a recent and fast growing field providing a setof new topological and geometric tools to infer relevant features for possiblycomplex data. This paper is a brief introduction, through a few selectedtopics, to basic fundamental and practical aspects of \tda\ for non experts. <<<

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, Ross Girshick <<<

We introduce the Segment Anything (SA) project: a new task, model, anddataset for image segmentation. Using our efficient model in a data collectionloop, we built the largest segmentation dataset to date (by far), with over 1billion masks on 11M licensed and privacy respecting images. The model isdesigned and trained to be promptable, so it can transfer zero-shot to newimage distributions and tasks. We evaluate its capabilities on numerous tasksand find that its zero-shot performance is impressive -- often competitive withor even superior to prior fully supervised results. We are releasing theSegment Anything Model (SAM) and corresponding dataset (SA-1B) of 1B masks and11M images at https://segment-anything.com to foster research into foundationmodels for computer vision. <<<

Diffusion models have significantly advanced the fields of image, audio, andvideo generation, but they depend on an iterative sampling process that causesslow generation. To overcome this limitation, we propose consistency models, anew family of models that generate high quality samples by directly mappingnoise to data. They support fast one-step generation by design, while stillallowing multistep sampling to trade compute for sample quality. They alsosupport zero-shot data editing, such as image inpainting, colorization, andsuper-resolution, without requiring explicit training on these tasks.Consistency models can be trained either by distilling pre-trained diffusionmodels, or as standalone generative models altogether. Through extensiveexperiments, we demonstrate that they outperform existing distillationtechniques for diffusion models in one- and few-step sampling, achieving thenew state-of-the-art FID of 3.55 on CIFAR-10 and 6.20 on ImageNet 64x64 forone-step generation. When trained in isolation, consistency models become a newfamily of generative models that can outperform existing one-step,non-adversarial generative models on standard benchmarks such as CIFAR-10,ImageNet 64x64 and LSUN 256x256. <<<

The rapid progress in the field of deep learning has had a significant impact on protein design. Deep learning methods have recently produced a breakthrough in protein structure prediction, leading to the availability of high-quality models for millions of proteins. Along with novel architectures for generative modeling and sequence analysis, they have revolutionized the protein design field in the past few years remarkably by improving the accuracy and ability to identify novel protein sequences and structures. Deep neural networks can now learn and extract the fundamental features of protein structures, predict how they interact with other biomolecules, and have the potential to create new effective drugs for treating disease. As their applicability in protein design is rapidly growing, we review the recent developments and technology in deep learning methods and provide examples of their performance to generate novel functional proteins. <<<

Qiming Sun, Xing Zhang, Samragni Banerjee, Peng Bao, Marc Barbry, Nick S Blunt, Nikolay A Bogdanov, George H Booth, Jia Chen, Zhi-Hao Cui, Janus J Eriksen, Yang Gao, Sheng Guo, Jan Hermann, Matthew R Hermes, Kevin Koh, Peter Koval, Susi Lehtola, Zhendong Li, Junzi Liu, Narbe Mardirossian, James D McClain, Mario Motta, Bastien Mussard, Hung Q Pham, Artem Pulkin, Wirawan Purwanto, Paul J Robinson, Enrico Ronca, Elvira R Sayfutyarova, Maximilian Scheurer, Henry F Schurkus, James E T Smith, Chong Sun, Shi-Ning Sun, Shiv Upadhyay, Lucas K Wagner, Xiao Wang, Alec White, James Daniel Whitfield, Mark J Williamson, Sebastian Wouters, Jun Yang, Jason M Yu, Tianyu Zhu, Timothy C Berkelbach, Sandeep Sharma, Alexander Yu Sokolov, Garnet Kin-Lic Chan <<<

PySCF is a Python-based general-purpose electronic structure platform that supports first-principles simulations of molecules and solids as well as accelerates the development of new methodology and complex computational workflows. This paper explains the design and philosophy behind PySCF that enables it to meet these twin objectives. With several case studies, we show how users can easily implement their own methods using PySCF as a development environment. We then summarize the capabilities of PySCF for molecular and solid-state simulations. Finally, we describe the growing ecosystem of projects that use PySCF across the domains of quantum chemistry, materials science, machine learning, and quantum information science. <<<

Recent events have pushed RNA research into the spotlight. Continued discoveries of RNA with unexpected diverse functions in healthy and diseased cells, such as the role of RNA as both the source and countermeasure to a severe acute respiratory syndrome coronavirus 2 infection, are igniting a new passion for understanding this functionally and structurally versatile molecule. Although RNA structure is key to function, many foundational characteristics of RNA structure are misunderstood, and the default state of RNA is often thought of and depicted as a single floppy strand. The purpose of this perspective is to help adjust mental models, equipping the community to better use the fundamental aspects of RNA structural information in new mechanistic models, enhance experimental design to test these models, and refine data interpretation. We discuss six core observations focused on the inherent nature of RNA structure and how to incorporate these characteristics to better understand RNA structure. We also offer some ideas for future efforts to make validated RNA structural information available and readily used by all researchers. <<<

Deep convolutional networks have demonstrated the state-of-the-art performance on various medical image computing tasks. Leveraging images from different modalities for the same analysis task holds clinical benefits. However, the generalization capability of deep models on test data with different distributions remain as a major challenge. In this paper, we propose the PnPAdaNet (plug-and-play adversarial domain adaptation network) for adapting segmentation networks between different modalities of medical images, e.g., MRI and CT. We propose to tackle the significant domain shift by aligning the feature spaces of source and target domains in an unsupervised manner. Specifically, a domain adaptation module flexibly replaces the early encoder layers of the source network, and the higher layers are shared between domains. With adversarial learning, we build two discriminators whose inputs are respectively multi-level features and predicted segmentation masks. We have validated our domain adaptation method on cardiac structure segmentation in unpaired MRI and CT. The experimental results with comprehensive ablation studies demonstrate the excellent efficacy of our proposed PnP-AdaNet. Moreover, we introduce a novel benchmark on the cardiac dataset for the task of unsupervised cross-modality domain adaptation. We will make our code and database publicly available, aiming to promote future studies on this challenging yet important research topic in medical imaging. <<<