For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation. Given a new sequence modeling task or dataset, which architecture should one use? We conduct a systematic evaluation of generic convolutional and recurrent architectures for sequence modeling. The models are evaluated across a broad range of standard tasks that are commonly used to benchmark recurrent networks. Our results indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets, while demonstrating longer effective memory. We conclude that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutional networks should be regarded as a natural starting point for sequence modeling tasks. To assist related work, we have made code available at this http URL . <<<

Jianming Yang, Geng Pei, Xuan Sun, Yawen Xiao, Chunhui Miao, Lu Zhou, Bangmao Wang, Liu Yang, Mingyu Yu, Zhi-Song Zhang, Evan T Keller, Zhi Yao, Quan Wang <<<

BACKGROUND: The pathogenesis of inflammatory bowel diseases (IBD) is multifactorial, and diagnostic and treatment strategies for IBD remain to be developed. RhoB regulates multiple cell functions; however, its role in colitis is unexplored.RESULTS: Here, we found RhoB was dramatically increased in colon tissues of ulcerative colitis (UC) patients and mice with DSS-induced colitis. Compared with wild type mice, RhoB+/- and RhoB-/- mice developed milder DSS-induced colitis and increased goblet cell numbers and IEC proliferation. Decreased RhoB promoted goblet cell differentiation and epithelial regeneration through inhibiting Wnt signaling pathway and activating p38 MAPK signaling pathway. Moreover, increased SCFA-producing bacteria and SCFA concentrations were detected in intestinal microbiome of both RhoB+/- and RhoB-/- mice and upregulated SCFA receptor expression was also observed.CONCLUSIONS: Taken together, a higher level of RhoB is associated with UC, which also contributes to UC development through modulating cell signaling and altering intestinal bacterial composition and metabolites. These observations suggest that RhoB has potential as a biomarker and a treatment target for UC. Video Abstract. <<<

In response to physical exercise and diet, skeletal muscle adapts to energetic demands through large transcriptional changes. This remodelling is associated with changes in skeletal muscle DNA methylation which may participate in the metabolic adaptation to extracellular stimuli. Yet, the mechanisms by which muscle-borne DNA methylation machinery responds to diet and exercise and impacts muscle function are unknown. Here, we investigated the function of de novo DNA methylation in fully differentiated skeletal muscle. We generated muscle-specific DNA methyltransferase 3A (DNMT3A) knockout mice (mD3AKO) and investigated the impact of DNMT3A ablation on skeletal muscle DNA methylation, exercise capacity and energy metabolism. Loss of DNMT3A reduced DNA methylation in skeletal muscle over multiple genomic contexts and altered the transcription of genes known to be influenced by DNA methylation, but did not affect exercise capacity and whole-body energy metabolism compared to wild type mice. Loss of DNMT3A did not alter skeletal muscle mitochondrial function or the transcriptional response to exercise however did influence the expression of genes involved in muscle development. These data suggest that DNMT3A does not have a large role in the function of mature skeletal muscle although a role in muscle development and differentiation is likely. <<<

Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos, Caiming Xiong, Zachary Z Sun, Richard Socher, James S Fraser, Nikhil Naik <<<

Deep-learning language models have shown promise in various biotechnological applications, including protein design and engineering. Here we describe ProGen, a language model that can generate protein sequences with a predictable function across large protein families, akin to generating grammatically and semantically correct natural language sentences on diverse topics. The model was trained on 280 million protein sequences from >19,000 families and is augmented with control tags specifying protein properties. ProGen can be further fine-tuned to curated sequences and tags to improve controllable generation performance of proteins from families with sufficient homologous samples. Artificial proteins fine-tuned to five distinct lysozyme families showed similar catalytic efficiencies as natural lysozymes, with sequence identity to natural proteins as low as 31.4%. ProGen is readily adapted to diverse protein families, as we demonstrate with chorismate mutase and malate dehydrogenase. <<<

State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at this https URL. <<<

An established normative approach for understanding the algorithmic basis of neural computation is to derive online algorithms from principled computational objectives and evaluate their compatibility with anatomical and physiological observations. Similarity matching objectives have served as successful starting points for deriving online algorithms that map onto neural networks (NNs) with point neurons and Hebbian/anti-Hebbian plasticity. These NN models account for many anatomical and physiological observations; however, the objectives have limited computational power and the derived NNs do not explain multi-compartmental neuronal structures and non-Hebbian forms of plasticity that are prevalent throughout the brain. In this article, we review and unify recent extensions of the similarity matching approach to address more complex objectives, including a broad range of unsupervised and self-supervised learning tasks that can be formulated as generalized eigenvalue problems or nonnegative matrix factorization problems. Interestingly, the online algorithms derived from these objectives naturally map onto NNs with multi-compartmental neurons and local, non-Hebbian learning rules. Therefore, this unified extension of the similarity matching approach provides a normative framework that facilitates understanding the multi-compartmental neuronal structures and non-Hebbian plasticity found throughout the brain. <<<

INTRODUCTION: The impact of pectoralis muscle mass index (PMI) on cardiac events is not well studied in cancer patients, especially in those who have received chemotherapy with high potential cardiac toxicity such as anthracyclines.METHODS: Individuals aged ≥18 years with a diagnosis of breast cancer, sarcoma, or lymphoma who received anthracycline-based chemotherapy at the University of Minnesota MHealth Fairview between 2009 and 2014. Eligible patients had to have two CT scans: a baseline CT scan within 6 months prior to chemotherapy and a follow-up CT scan within 2 years after treatment. The PMI was calculated as the right pectoralis muscle area indexed to height squared. Multivariable linear regression was used to analyze factors associated with PMI at follow-up, overall mortality, and major cardiac events (MACE).RESULTS: A total of 474 patients (breast cancer 192; lymphoma 184; sarcoma 98) participated with a median age of 61 years at the time of baseline CT scan; 161 (34%) were male. Almost all patients received anthracyclines except 12% who received trastuzumab only. The median baseline PMI was 5.8 cm2/m2 (4.9, 7.7) which decreased 10.5% after chemotherapy, to 5.2 cm2/m2 (4.4, 6.4). Baseline PMI was not significantly associated with OS, but we detected lower risks of MACE with larger PMI at baseline. Greater baseline PMI was associated with greater follow-up PMI, but also with greater relative PMI loss. Female gender, older age, and history of smoking were also associated with greater PMI losses.CONCLUSION: Greater pre-treatment pectoralis muscle index in patients treated with anthracyclines have a lower risk of MACE. Early identification of sarcopenia using PMI could trigger proactive engagement for intervention and risk-stratified therapies. <<<

Perceptual events derive their significance to an animal from their meaning about the world, that is from the information they carry about their causes. The brain should thus be able to efficiently infer the causes underlying our sensory events. Here we use multisensory cue combination to study causal inference in perception. We formulate an ideal-observer model that infers whether two sensory cues originate from the same location and that also estimates their location(s). This model accurately predicts the nonlinear integration of cues by human subjects in two auditory-visual localization tasks. The results show that indeed humans can efficiently infer the causal structure as well as the location of causes. By combining insights from the study of causal inference with the ideal-observer approach to sensory cue combination, we show that the capacity to infer causal structure is not limited to conscious, high-level cognition; it is also performed continually and effortlessly in perception. <<<

The role of enhancers, a key class of non-coding regulatory DNA elements, in cancer development has increasingly been appreciated. Here, we present the detection and characterization of a large number of expressed enhancers in a genome-wide analysis of 8928 tumor samples across 33 cancer types using TCGA RNA-seq data. Compared with matched normal tissues, global enhancer activation was observed in most cancers. Across cancer types, global enhancer activity was positively associated with aneuploidy, but not mutation load, suggesting a hypothesis centered on "chromatin-state" to explain their interplay. Integrating eQTL, mRNA co-expression, and Hi-C data analysis, we developed a computational method to infer causal enhancer-gene interactions, revealing enhancers of clinically actionable genes. Having identified an enhancer ∼140 kb downstream of PD-L1, a major immunotherapy target, we validated it experimentally. This study provides a systematic view of enhancer activity in diverse tumor contexts and suggests the clinical implications of enhancers. <<<

The effects of pasteurisation, high-pressure (HP), and a combination of pasteurisation and high-pressure (PHP) on the physicochemical properties and protein structure of caprine skim milk was investigated. Samples treated by PHP generally had a higher pH, whey protein denaturation, surface hydrophobicity, and intrinsic fluorescence than those treated only with heat or pressure. In contrast, the size of skim milk casein micelles decreased significantly with an increase in pressure level and time; however, the effect was less marked when heat and pressure treatments were combined. For the PHP and HP samples, as the level and time of pressure increased, the α-helix and β-turn content reduced, whereas β-sheet and random coil were induced. Thus, PHP treatment could be used as a good alternative technology in the dairy industry to promote the functional properties of skim caprine milk. <<<

The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to be effective on a variety of tasks; however, no LLM specialized for the financial domain has been reported in literature. In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet, augmented with 345 billion tokens from general purpose datasets. We validate BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite of internal benchmarks that most accurately reflect our intended usage. Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. Additionally, we explain our modeling choices, training process, and evaluation methodology. As a next step, we plan to release training logs (Chronicles) detailing our experience in training BloombergGPT. <<<

OBJECTIVES: Bacteriome and virome alterations are associated with colorectal cancer (CRC). Nevertheless, the gut fungal microbiota in CRC remains largely unexplored. We aimed to characterise enteric mycobiome in CRC.DESIGN: Faecal shotgun metagenomic sequences of 184 patients with CRC, 197 patients with adenoma and 204 control subjects from Hong Kong were analysed (discovery cohort: 73 patients with CRC and 92 control subjects; validation cohort: 111 patients with CRC, 197 patients with adenoma and 112 controls from Hong Kong). CRC-associated fungal markers and ecological changes were also validated in additional independent cohorts of 90 patients with CRC, 42 patients with adenoma and 66 control subjects of published repository sequences from Germany and France. Assignment of taxonomies was performed by exact k-mer alignment against an integrated microbial reference genome database.RESULTS: Principal component analysis revealed separate clusters for CRC and control (p<0.0001), with distinct mycobiomes in early-stage and late-stage CRC (p=0.0048). Basidiomycota:Ascomycota ratio was higher in CRC (p=0.0042), with increase in Malasseziomycetes (p<0.0001) and decrease in Saccharomycetes (p<0.0001) and Pneumocystidomycetes (p=0.0017). Abundances of 14 fungal biomarkers distinguished CRC from controls with an area under the receiver-operating characteristic curve (AUC) of 0.93 and validated AUCs of 0.82 and 0.74 in independent Chinese cohort V1 and European cohort V2, respectively. Further ecological analysis revealed higher numbers of co-occurring fungal intrakingdom and co-exclusive bacterial-fungal correlations in CRC (p<0.0001). Moreover, co-occurrence interactions between fungi and bacteria, mostly contributed by fungal Ascomycota and bacterial Proteobacteria in control, were reverted to co-exclusive interplay in CRC (p=0.00045).CONCLUSIONS: This study revealed CRC-associated mycobiome dysbiosis characterised by altered fungal composition and ecology, signifying that the gut mycobiome might play a role in CRC. <<<

Hyperglycemia, or elevated blood glucose, renders individuals more prone to developing severe Staphylococcus aureus infections. S. aureus is the most common etiological agent of musculoskeletal infection, which is a common manifestation of disease in hyperglycemic patients. However, the mechanisms by which S. aureus causes severe musculoskeletal infection during hyperglycemia are incompletely characterized. To examine the influence of hyperglycemia on S. aureus virulence during invasive infection, we used a murine model of osteomyelitis and induced hyperglycemia with streptozotocin. We discovered that hyperglycemic mice exhibited increased bacterial burdens in bone and enhanced dissemination compared to control mice. Furthermore, infected hyperglycemic mice sustained increased bone destruction relative to euglycemic controls, suggesting that hyperglycemia exacerbates infection-associated bone loss. To identify genes contributing to S. aureus pathogenesis during osteomyelitis in hyperglycemic animals relative to euglycemic controls, we used transposon sequencing (TnSeq). We identified 71 genes uniquely essential for S. aureus survival in osteomyelitis in hyperglycemic mice and another 61 mutants with compromised fitness. Among the genes essential for S. aureus survival in hyperglycemic mice was the gene encoding superoxide dismutase A (), one of two S. aureus superoxide dismutases involved in detoxifying reactive oxygen species (ROS). We determined that a mutant exhibits attenuated survival in high glucose and during osteomyelitis in hyperglycemic mice. SodA therefore plays an important role during growth in high glucose and promotes S. aureus survival in bone. Collectively, these studies demonstrate that hyperglycemia increases the severity of osteomyelitis and identify genes contributing to S. aureus survival during hyperglycemic infection. <<<

Pattern separation, or the process by which highly similar stimuli or experiences in memory are represented by non-overlapping neural ensembles, has typically been ascribed to processes supported by the hippocampus. Converging evidence from a wide range of studies, however, suggests that pattern separation is a multistage process supported by a network of brain regions. Based on this evidence, considered together with related findings from the interference resolution literature, we propose the 'cortico-hippocampal pattern separation' (CHiPS) framework, which asserts that brain regions involved in cognitive control play a significant role in pattern separation. Particularly, these regions may contribute to pattern separation by (1) resolving interference in sensory regions that project to the hippocampus, thus regulating its cortical input, or (2) directly modulating hippocampal processes in accordance with task demands. Considering recent interest in how hippocampal operations are modulated by goal states likely represented and regulated by extra-hippocampal regions, we argue that pattern separation is similarly supported by neocortical-hippocampal interactions. <<<

Hongxi Zhang, Jia Li, Xiaoli Su, Yang Hu, Tianmei Liu, Shaoqing Ni, Haifeng Li, Xi-Nian Zuo, Junfen Fu, Ti-Fei Yuan, Zhi Yang <<<

Brain development from 1 to 6 years of age anchors a wide range of functional capabilities and carries early signs of neurodevelopmental disorders. However, quantitative models for depicting brain morphology changes and making individualized inferences are lacking, preventing the identification of early brain atypicality during this period. With a sample size of 285, we characterized the age dependence of the cortical thickness and subcortical volume in neurologically normal children and constructed quantitative growth charts of all brain regions for preschool children. While the cortical thickness of most brain regions decreased with age, the entorhinal and parahippocampal regions displayed an inverted-U shape of age dependence. Compared to the cortical thickness, the normalized volume of subcortical regions exhibited more divergent trends, with some regions increasing, some decreasing, and some displaying inverted-U-shaped trends. The growth curve models for all brain regions demonstrated utilities in identifying brain atypicality. The percentile measures derived from the growth curves facilitate the identification of children with developmental speech and language disorders with an accuracy of 0.875 (area under the receiver operating characteristic curve: 0.943). Our results fill the knowledge gap in brain morphometrics in a critical development period and provide an avenue for individualized brain developmental status evaluation with demonstrated sensitivity. The brain growth charts are shared with the public (http://phi-group.top/resources.html). <<<

Keren Yizhak, François Aguet, Jaegil Kim, Julian M Hess, Kirsten Kübler, Jonna Grimsby, Ruslana Frazer, Hailei Zhang, Nicholas J Haradhvala, Daniel Rosebrock, Dimitri Livitz, Xiao Li, Eila Arich-Landkof, Noam Shoresh, Chip Stewart, Ayellet V Segrè, Philip A Branton, Paz Polak, Kristin G Ardlie, Gad Getz <<<

How somatic mutations accumulate in normal cells is poorly understood. A comprehensive analysis of RNA sequencing data from ~6700 samples across 29 normal tissues revealed multiple somatic variants, demonstrating that macroscopic clones can be found in many normal tissues. We found that sun-exposed skin, esophagus, and lung have a higher mutation burden than other tested tissues, which suggests that environmental factors can promote somatic mosaicism. Mutation burden was associated with both age and tissue-specific cell proliferation rate, highlighting that mutations accumulate over both time and number of cell divisions. Finally, normal tissues were found to harbor mutations in known cancer genes and hotspots. This study provides a broad view of macroscopic clonal expansion in human tissues, thus serving as a foundation for associating clonal expansion with environmental factors, aging, and risk of disease. <<<

Maxime Meylan, Florent Petitprez, Etienne Becht, Antoine Bougoüin, Guilhem Pupier, Anne Calvez, Ilenia Giglioli, Virginie Verkarre, Guillaume Lacroix, Johanna Verneau, Chen-Ming Sun, Pierre Laurent-Puig, Yann-Alexandre Vano, Reza Elaïdi, Arnaud Méjean, Rafaël Sanchez-Salas, Eric Barret, Xavier Cathelineau, Stephane Oudard, Claude-Agnès Reynaud, Aurélien de Reyniès, Catherine Sautès-Fridman, Wolf Herman Fridman <<<

The presence of intratumoral tertiary lymphoid structures (TLS) is associated with positive clinical outcomes and responses to immunotherapy in cancer. Here, we used spatial transcriptomics to examine the nature of B cell responses within TLS in renal cell carcinoma (RCC). B cells were enriched in TLS, and therein, we could identify all B cell maturation stages toward plasma cell (PC) formation. B cell repertoire analysis revealed clonal diversification, selection, expansion in TLS, and the presence of fully mature clonotypes at distance. In TLS+ tumors, IgG- and IgA-producing PCs disseminated into the tumor beds along fibroblastic tracks. TLS+ tumors exhibited high frequencies of IgG-producing PCs and IgG-stained and apoptotic malignant cells, suggestive of anti-tumor effector activity. Therapeutic responses and progression-free survival correlated with IgG-stained tumor cells in RCC patients treated with immune checkpoint inhibitors. Thus, intratumoral TLS sustains B cell maturation and antibody production that is associated with response to immunotherapy, potentially via direct anti-tumor effects. <<<

Luxi Chen, Jing Li, Renqiang Yuan, Yujie Wang, Jiaman Zhang, Yu Lin, Lina Wang, Xingxing Zhu, Wei Zhu, Jingyi Bai, Fanli Kong, Bo Zeng, Lu Lu, Jideng Ma, Keren Long, Long Jin, Zhiqing Huang, Jinlong Huo, Yiren Gu, Danyang Wang, Delin Mo, Diyan Li, Qianzi Tang, Xuewei Li, Jiangwei Wu, Yaosheng Chen, Mingzhou Li <<<

Liver development is a complex process that is regulated by a series of signaling pathways. Three-dimensional (3D) chromatin architecture plays an important role in transcriptional regulation; nonetheless, its dynamics and role in the rapid transition of core liver functions during development and obesity-induced metabolic stress remain largely unexplored. To investigate the dynamic chromatin architecture during liver development and under metabolic stress, we generated high-resolution maps of chromatin architecture for porcine livers across six major developmental stages (from embryonic day 38 to the adult stage) and under a high-fat diet-induced obesity. The characteristically loose chromatin architecture supports a highly plastic genome organization during early liver development, which fundamentally contributes to the rapid functional transitions in the liver after birth. We reveal the multi-scale reorganization of chromatin architecture and its influence on transcriptional regulation of critical signaling processes during liver development, and show its close association with transition in hepatic functions (i.e., from hematopoiesis in the fetus to metabolism and immunity after birth). The limited changes in chromatin structure help explain the observed metabolic adaptation to excessive energy intake in pigs. These results provide a global overview of chromatin architecture dynamics associated with the transition of physiological liver functions between prenatal development and postnatal maturation, and a foundational resource that allows for future in-depth functional characterization. <<<

Gamma oscillations are thought to coordinate the spike timing of functionally specialized neuronal ensembles across brain regions. To test this hypothesis, we optogenetically perturbed gamma spike timing in the rat medial (MEC) and lateral (LEC) entorhinal cortices and found impairments in spatial and object learning tasks, respectively. MEC and LEC were synchronized with the hippocampal dentate gyrus through high- and low-gamma-frequency rhythms, respectively, and engaged either granule cells or mossy cells and CA3 pyramidal cells in a task-dependent manner. Gamma perturbation disrupted the learning-induced assembly organization of target neurons. Our findings imply that pathway-specific gamma oscillations route task-relevant information between distinct neuronal subpopulations in the entorhinal-hippocampal circuit. We hypothesize that interregional gamma-time-scale spike coordination is a mechanism of neuronal communication. <<<