Huakang Li, Zixuan Zhao, Meiling Shi, Bin Luo, Guangdong Wang, Xiaojuan Wang, Jie Gu, Zilin Song, Yifan Sun, Li Zhang, Jia Wang <<<

Position encoding recently has shown effective in the transformer architecture. It enables valuable supervision for dependency modeling between elements at different positions of the sequence. In this paper, we first investigate various methods to integrate positional information into the learning process of transformer-based language models. Then, we propose a novel method named Rotary Position Embedding(RoPE) to effectively leverage the positional information. Specifically, the proposed RoPE encodes the absolute position with a rotation matrix and meanwhile incorporates the explicit relative position dependency in self-attention formulation. Notably, RoPE enables valuable properties, including the flexibility of sequence length, decaying inter-token dependency with increasing relative distances, and the capability of equipping the linear self-attention with relative position encoding. Finally, we evaluate the enhanced transformer with rotary position embedding, also called RoFormer, on various long text classification benchmark datasets. Our experiments show that it consistently overcomes its alternatives. Furthermore, we provide a theoretical analysis to explain some experimental results. RoFormer is already integrated into Huggingface: \url{https://huggingface.co/docs/transformers/model_doc/roformer}. <<<

Shoupu He, Gaofei Sun, Xiaoli Geng, Wenfang Gong, Panhong Dai, Yinhua Jia, Weijun Shi, Zhaoe Pan, Junduo Wang, Liyuan Wang, Songhua Xiao, Baojun Chen, Shufang Cui, Chunyuan You, Zongming Xie, Feng Wang, Jie Sun, Guoyong Fu, Zhen Peng, Daowu Hu, Liru Wang, Baoyin Pang, Xiongming Du <<<

Abstract Diffusion probabilistic models (DPMs) have achieved impressive success in high-resolution image synthesis, especially in recent large-scale text-to-image generation applications. An essential technique for improving the sample quality of DPMs is guided sampling, which usually needs a large guidance scale to obtain the best sample quality. The commonly-used fast sampler for guided sampling is denoising diffusion implicit models (DDIM), a first-order diffusion ordinary differential equation (ODE) solver that generally needs 100 to 250 steps for high-quality samples. Although recent works propose dedicated high-order solvers and achieve a further speedup for sampling without guidance, their effectiveness for guided sampling has not been well-tested before. In this work, we demonstrate that previous high-order fast samplers suffer from instability issues, and they even become slower than DDIM when the guidance scale grows larger. To further speed up guided sampling, we propose DPM-Solver++, a high-order solver for the guided sampling of DPMs. DPM-Solver++ solves the diffusion ODE with the data prediction model and adopts thresholding methods to keep the solution matches training data distribution. We further propose a multistep variant of DPM-Solver++ to address the instability issue by reducing the effective step size. Experiments show that DPM-Solver++ can generate high-quality samples within only 15 to 20 steps for guided sampling by pixel-space and latent-space DPMs. <<<

1 Abstract Interactions between proteins and other biomolecules underlie nearly all biological processes, yet designing such interactions de novo remains challenging. Capturing their specific interactions and co-optimizing sequence and structure are difficult and often require extensive computation. We present Protein Hunter, a fast, fine-tuning-free framework for de novo protein design. Starting from an all-X sequence, we find diffusion-based structure prediction models hallucinate reasonable looking structures that can be further improved through iterative sequence re-design and structure re-prediction. This lightweight strategy achieves high AlphaFold3 in silico success rates across both unconditional and conditional generation tasks, including binders to proteins, cyclic peptides, small molecules, DNA, and RNA. Protein Hunter also supports multi-motif scaffolding and partial redesign, providing a general and efficient platform for de novo protein design across diverse molecular targets. <<<

Hui Wang, Yuxin Chen, Jun Lv, Xiaoting Cheng, Qi Cao, Daqi Wang, Longlong Zhang, Biyun Zhu, Min Shen, Chunxin Xu, Mengzhao Xun, Zijing Wang, Honghai Tang, Shaowei Hu, Chong Cui, Luoying Jiang, Yanbo Yin, Luo Guo, Yi Zhou, Lei Han, Ziwen Gao, Jiajia Zhang, Sha Yu, Kaiyu Gao, Jinghan Wang, Bing Chen, Wuqing Wang, Zheng-Yi Chen, Huawei Li, Yilai Shu <<<

AbstractGene therapy is a promising approach for hereditary deafness. We recently showed that unilateral AAV1-hOTOF gene therapy with dual adeno-associated virus (AAV) serotype 1 carrying human OTOF transgene is safe and associated with functional improvements in patients with autosomal recessive deafness 9 (DFNB9). The protocol was subsequently amended and approved to allow bilateral gene therapy administration. Here we report an interim analysis of the single-arm trial investigating the safety and efficacy of binaural therapy in five pediatric patients with DFNB9. The primary endpoint was dose-limiting toxicity at 6 weeks, and the secondary endpoint included safety (adverse events) and efficacy (auditory function and speech perception). No dose-limiting toxicity or serious adverse event occurred. A total of 36 adverse events occurred. The most common adverse events were increased lymphocyte counts (6 out of 36) and increased cholesterol levels (6 out of 36). All patients had bilateral hearing restoration. The average auditory brainstem response threshold in the right (left) ear was >95 dB (>95 dB) in all patients at baseline, and the average auditory brainstem response threshold in the right (left) ear was restored to 58 dB (58 dB) in patient 1, 75 dB (85 dB) in patient 2, 55 dB (50 dB) in patient 3 at 26 weeks, and 75 dB (78 dB) in patient 4 and 63 dB (63 dB) in patient 5 at 13 weeks. The speech perception and the capability of sound source localization were restored in all five patients. These results provide preliminary insights on the safety and efficacy of binaural AAV gene therapy for hereditary deafness. The trial is ongoing with longer follow-up to confirm the safety and efficacy findings. Chinese Clinical Trial Registry registration: ChiCTR2200063181. <<<

Victor R. Ambros, Martin Chalfie, Aric L. Daul, Andrew Z. Fire, David H. Hall, H. Robert Horvitz, Craig C. Mello, Gary Ruvkun, Nathan E. Schroeder, Paul W. Sternberg, Ann E. Rougvie <<<

Experimental organisms such as the nematode Caenorhabditis elegans are fundamental to biological discovery. The success of C. elegans research has been greatly enabled by infrastructure that allows thousands of scientists to share and access research materials and unpublished information efficiently. Here, we celebrate the worm by interweaving vignettes describing four Nobel Prize–winning discoveries with descriptions of how the major NIH-supported research resources—the Caenorhabditis Genetics Center, WormBase, and WormAtlas—provide invaluable support for all C. elegans research. The synergy between investigation and the availability of shared resources for the C. elegans community is a paradigm for all model organism research, and the continued support of such community research resources will be essential for maximizing impactful discoveries in the future. <<<

Hojae Choi, Brian Coventry, Magnus Bauer, Preetham Venkatesh, Anqi Chen, Donghyo Kim, Asim K. Bera, Alex Kang, Hannah Nguyen, Emily Joyce, Bhanumathi Shankaran, Tuscan Rock Thompson, Jacob Merle Gershon, Alexander F. Shida, Gyu Rie Lee, Donald Hilvert, Samuel J. Pellock, David Baker <<<

Abstract Despite advances in de novo enzyme design, success has been largely limited to low energy barrier model reactions. Amide bonds such as those linking amino acids along the peptide backbone are stable for hundreds of years in neutral aqueous solution because of the high energy barrier to hydrolysis 1 . Here we describe the use of a new deep learning method, RFD2-MI 2 , to de novo design enzymes which utilize an activated cysteine nucleophile to hydrolyze the polypeptide backbone in a sequence-dependent manner, achieving rate enhancements over the background reaction ( k cat / k uncat ) of up to 3 × 10 7 . The generated designs have folds very different from the proteases in nature (TM score < 0.50), and crystal structures are very close to the design models (Cα RMSDs < 1.2 Å), highlighting the accuracy of the design methodology. Our approach has broad utility for advancing the design of novel proteases for both biotechnical and medical applications. <<<

The intricate neurofluid dynamics and balance is essential in preserving the structural and functional integrity of the brain. Key among these forces are: hemodynamics, such as heartbeat-driven arterial and venous blood flow, and hydrodynamics, such as cerebrospinal fluid (CSF) circulation. The delicate interplay between these dynamics is crucial for maintaining optimal homeostasis within the brain. Currently, the widely accepted framework for understanding brain functions is the Monro-Kellie’s doctrine, which posits a constant sum of intracranial CSF, blood flow and brain tissue volumes. However, in recent decades, there has been a growing interest in exploring the dynamic interplay between these elements and the impact of external factors, such as daily changes in body position. CSF circulation in particular plays a crucial role in the context of neurodegeneration and dementia, since its dysfunction has been associated with impaired clearance mechanisms and accumulation of toxic substances. Despite the implementation of various invasive and non-invasive imaging techniques to investigate the intracranial hemodynamic or hydrodynamic properties, a comprehensive understanding of how all these elements interact and are influenced by body position remains wanted. Establishing a comprehensive overview of this topic is therefore crucial and could pave the way for alternative care approaches. In this review, we aim to summarize the existing understanding of intracranial hemodynamic and hydrodynamic properties, fundamental for brain homeostasis, along with factors known to influence their equilibrium. Special attention will be devoted to elucidating the effects of body position shifts, given their significance and remaining ambiguities. Furthermore, we will explore recent advancements in imaging techniques utilized for real time and non-invasive measurements of dynamic body fluid properties in-vivo. <<<

An autonomous robotic nut tightening system for a serial manipulator equipped with a parallel gripper is proposed. The system features a hierarchical motion-primitive-based planner and a control-switching scheme that alternates between force and position control. Extensive simulations demonstrate the system's robustness to variance in initial conditions. Additionally, the proposed controller tightens threaded screws 14% faster than the baseline while applying 40 times less contact force on manipulands. For the benefit of the research community, the system's implementation is open-sourced. <<<

Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, Alexander Rives <<<

Recent advances in machine learning have leveraged evolutionary information in multiple sequence alignments to predict protein structure. We demonstrate direct inference of full atomic-level protein structure from primary sequence using a large language model. As language models of protein sequences are scaled up to 15 billion parameters, an atomic-resolution picture of protein structure emerges in the learned representations. This results in an order-of-magnitude acceleration of high-resolution structure prediction, which enables large-scale structural characterization of metagenomic proteins. We apply this capability to construct the ESM Metagenomic Atlas by predicting structures for >617 million metagenomic protein sequences, including >225 million that are predicted with high confidence, which gives a view into the vast breadth and diversity of natural proteins. <<<

Jason Guo, Roger Liang, Andrew Chung, Zhijie Li, Boyuan Li, Eric Chen, Lin Li, Jingjing Wang, Meng-Hsiung Hsieh, Ivy Xiangyi Fang, Benjamin Kroger, Yunguan Wang, Min Zhu, Xiongzhao Ren, Greg Mannino, Yuemeng Jia, Yonglong Wei, Stephen Moore, Daniel J. Siegwart, Stephen S. Chung, Zixi Wang, Tripti Sharma, Suman Komjeti, Yi Han, Purva Gopal, Guanghua Xiao, Tao Wang, Hao Zhu <<<

The origin of cancer is poorly understood because premalignant cells are rarely followed in their native environments. While the spatial compartmentalization of metabolic functions is critical for proper liver function, it is unknown if cancers arise from some zones but not others, and if there are metabolic determinants of cancer risk. Zone-specific, mosaic introduction of Ctnnb1 and Arid2 mutations, commonly co-mutated genes in hepatocellular carcinoma (HCC), showed that position and metabolic context determine clone fates. Ctnnb1/Arid2 -driven cancers were much more likely to arise in zone 3. The zone 3 genes Gstm2 and Gstm3 were required for efficient HCC initiation, in part through inhibition of ferroptosis. In the liver, the zonal determinants of HCC development can reveal metabolic vulnerabilities of cancer. <<<

How are systems supporting high-level cognition organized in the human brain? We hypothesize that cognitive processes involved in understanding people and places are implemented by distinct neural systems with parallel anatomical organization. We test this hypothesis using precision neuroimaging of individual human brains on diverse tasks involving perception and cognition in the domains of familiar people, places, and objects. We find that thinking about people and places elicits responses in distinct areas of high-level association cortex within the default mode network, spanning the frontal, parietal, and temporal lobes. Person- and place-preferring brain regions are systematically spatially adjacent across cortical zones. These areas have strongly domain-specific response profiles across visual, semantic, and episodic tasks and are specifically functionally connected to other parts of association cortex with like domain preference. Social and spatial networks remain anatomically separated at the apex of a unimodal-to-transmodal gradient across cortex and include regions with anatomical connections to the hippocampal formation. These results demonstrate the existence of parallel, domain-specific networks reaching the cortical apex. <<<

Li Wang, Cheng Wang, Juan A. Moriano, Songcang Chen, Guolong Zuo, Arantxa Cebrián-Silla, Shaobo Zhang, Tanzila Mukhtar, Shaohui Wang, Mengyi Song, Lilian Gomes de Oliveira, Qiuli Bi, Jonathan J. Augustin, Xinxin Ge, Mercedes F. Paredes, Eric J. Huang, Arturo Alvarez-Buylla, Xin Duan, Jingjing Li, Arnold R. Kriegstein <<<