基于物理信息的神经网络(Physics-informed Neural Networks,PINN),是一类用于解决有监督学习任务的神经网络,它不仅尽力遵循训练数据样本的分布规律,而且遵守由偏微分方程描述的物理定律。与纯数据驱动的神经网络学习相比,PINN在训练过程中施加了物理信息约束,因此能用更少的数据样本学习得到更具泛化能力的模型。近年来,PINN已逐渐成为机器学习和计算数学交叉学科的研究热点,并在理论和应用方面都获得了相对深入的研究,取得了可观的进展。但PINN独特的网络结构在实际应用中也存在训练缓慢甚至不收敛、精度低等问题。文中在总结当前PINN研究的基础上,对其网络/体系设计及其在流体力学等多个领域中的应用进行了探究,并展望了进一步的研究方向。 <<<

In the context of science, the well-known adage "a picture is worth a thousand words" might well be "a model is worth a thousand datasets." In this manuscript we introduce the SciML software ecosystem as a tool for mixing the information of physical laws and scientific models with data-driven machine learning approaches. We describe a mathematical object, which we denote universal differential equations (UDEs), as the unifying framework connecting the ecosystem. We show how a wide variety of applications, from automatically discovering biological mechanisms to solving high-dimensional Hamilton-Jacobi-Bellman equations, can be phrased and efficiently handled through the UDE formalism and its tooling. We demonstrate the generality of the software tooling to handle stochasticity, delays, and implicit constraints. This funnels the wide variety of SciML applications into a core set of training mechanisms which are highly optimized, stabilized for stiff equations, and compatible with distributed parallelism and GPU accelerators. <<<

In this work we investigate the use of the Signature Transform in the context of Learning. Under this assumption, we advance a supervised framework that potentially provides state-of-the-art classification accuracy with the use of few labels without the need of credit assignment and with minimal or no overfitting. We leverage tools from harmonic analysis by the use of the signature and log-signature, and use as a score function RMSE and MAE Signature and log-signature. We develop a closed-form equation to compute probably good optimal scale factors, as well as the formulation to obtain them by optimization. Techniques of Signal Processing are addressed to further characterize the problem. Classification is performed at the CPU level orders of magnitude faster than other methods. We report results on AFHQ, MNIST and CIFAR10, achieving 100% accuracy on all tasks assuming we can determine at test time which probably good optimal scale factor to use for each category. <<<

BACKGROUND: Tumor response to therapy is affected by both the cell types and the cell states present in the tumor microenvironment. This is true for many cancer treatments, including immune checkpoint inhibitors (ICIs). While it is well-established that ICIs promote T cell activation, their broader impact on other intratumoral immune cells is unclear; this information is needed to identify new mechanisms of action and improve ICI efficacy. Many preclinical studies have begun using single-cell analysis to delineate therapeutic responses in individual immune cell types within tumors. One major limitation to this approach is that therapeutic mechanisms identified in preclinical models have failed to fully translate to human disease, restraining efforts to improve ICI efficacy in translational research.METHOD: We previously developed a computational transfer learning approach called projectR to identify shared biology between independent high-throughput single-cell RNA-sequencing (scRNA-seq) datasets. In the present study, we test this algorithm's ability to identify conserved and clinically relevant transcriptional changes in complex tumor scRNA-seq data and expand its application to the comparison of scRNA-seq datasets with additional data types such as bulk RNA-seq and mass cytometry.RESULTS: We found a conserved signature of NK cell activation in anti-CTLA-4 responsive mouse and human tumors. In human metastatic melanoma, we found that the NK cell activation signature associates with longer overall survival and is predictive of anti-CTLA-4 (ipilimumab) response. Additional molecular approaches to confirm the computational findings demonstrated that human NK cells express CTLA-4 and bind anti-CTLA-4 antibodies independent of the antibody binding receptor (FcR) and that similar to T cells, CTLA-4 expression by NK cells is modified by cytokine-mediated and target cell-mediated NK cell activation.CONCLUSIONS: These data demonstrate a novel application of our transfer learning approach, which was able to identify cell state transitions conserved in preclinical models and human tumors. This approach can be adapted to explore many questions in cancer therapeutics, enhance translational research, and enable better understanding and treatment of disease. <<<

随着人均预期寿命的延长和受教育年限的提高,包括中国在内的世界各国劳动者的实际退休年龄却普遍低于法定退休年龄,呈现提前退休趋势。本文试图在生育受到约束的制度环境下对上述反常现象进行理论解释,通过构建一个动态一般均衡世代交叠(OLG)模型,考察预期寿命延长对劳动者的人力资本投资、退休年龄选择的影响机制,并结合中国的现实经济进行数值模拟。研究表明:当预期寿命提高时,人力资本投资能够在生命周期中获得更高的工资率回报,劳动者在少年期倾向于进行更多的人力资本投资,提高受教育年限;人力资本积累、有效工资率上升和利率下降引起的收入效应超过了替代效应,使得劳动者在老年期增加对闲暇的需求,有能力和意愿提早退休,减少终生劳动供给时间;进一步地,本文发现放松生育控制政策也会使劳动者享受更多闲暇时间的意愿增强,选择提前退休。本文还考察了个体退休行为的异质性,发现随着预期寿命的延长,相对穷人而言,富人的退休年龄和终生劳动供给更低。本文的结论有益于厘清劳动者在老年期的退休决策机理,为如何推进退休制度改革提供理论依据。 <<<

Wu Xiong, Alexandre Jousset, Rong Li, Manuel Delgado-Baquerizo, Mohammad Bahram, Ramiro Logares, Benjamin Wilden, Gerard Arjen de Groot, Nathalie Amacker, George A Kowalchuk, Qirong Shen, Stefan Geisen <<<

The colossal project of mapping the microbiome on Earth is rapidly advancing, with a focus on individual microbial groups. However, a global assessment of the associations between predatory protists and their bacterial prey is still missing at a cross-ecosystem level. This knowledge is critical to better understand the importance of top-down links in structuring microbiomes. Here, we examined 38 sequence-based datasets of paired bacterial and protistan taxa, covering 3,178 samples from diverse habitats including freshwater, marine and soils. We show that community profiles of protists and bacteria strongly correlated across and within habitats, with trophic microbiome structures fundamentally differing across habitats. Soils hosted the most heterogenous and diverse microbiomes. Protist communities were dominated by predators in soils and phototrophs in aquatic environments. This led to changes in the ratio of total protists to bacteria richness, which was highest in marine, while that of predatory protists to bacteria was highest in soils. Taxon richness and relative abundance of predatory protists positively correlated with bacterial richness in marine habitats. These links differed between soils, predatory protist richness and the relative abundance of predatory protists positively correlated with bacterial richness in forest and grassland soils, but not in agricultural soils. Our results suggested that anthropogenic pressure affects higher trophic levels more than lower ones leading to a decoupled trophic structure in microbiomes. Together, our cumulative overview of microbiome patterns of bacteria and protists at the global scale revealed major patterns and differences of the trophic structure of microbiomes across Earth's habitats, and show that anthropogenic factors might have negative effects on the trophic structure within microbiomes. Furthermore, the increased impact of anthropogenic factors on especially higher trophic levels suggests that often-observed reduced ecosystem functions in anthropogenic systems might be partly attributed to a reduction of trophic complexity. <<<

In coming to understand the world-in learning concepts, acquiring language, and grasping causal relations-our minds make inferences that appear to go far beyond the data available. How do we do it? This review describes recent approaches to reverse-engineering human learning and cognitive development and, in parallel, engineering more humanlike machine learning systems. Computational models that perform probabilistic inference over hierarchies of flexibly structured representations can address some of the deepest questions about the nature and origins of human thought: How does abstract knowledge guide learning and reasoning from sparse data? What forms does our knowledge take, across different domains and tasks? And how is that abstract knowledge itself acquired? <<<

V Gopalakrishnan, C N Spencer, L Nezi, A Reuben, M C Andrews, T V Karpinets, P A Prieto, D Vicente, K Hoffman, S C Wei, A P Cogdill, L Zhao, C W Hudgens, D S Hutchinson, T Manzo, M Petaccia de Macedo, T Cotechini, T Kumar, W S Chen, S M Reddy, R Szczepaniak Sloane, J Galloway-Pena, H Jiang, P L Chen, E J Shpall, K Rezvani, A M Alousi, R F Chemaly, S Shelburne, L M Vence, P C Okhuysen, V B Jensen, A G Swennes, F McAllister, E Marcelo Riquelme Sanchez, Y Zhang, E Le Chatelier, L Zitvogel, N Pons, J L Austin-Breneman, L E Haydu, E M Burton, J M Gardner, E Sirmans, J Hu, A J Lazar, T Tsujikawa, A Diab, H Tawbi, I C Glitza, W J Hwu, S P Patel, S E Woodman, R N Amaria, M A Davies, J E Gershenwald, P Hwu, J E Lee, J Zhang, L M Coussens, Z A Cooper, P A Futreal, C R Daniel, N J Ajami, J F Petrosino, M T Tetzlaff, P Sharma, J P Allison, R R Jenq, J A Wargo <<<

Preclinical mouse models suggest that the gut microbiome modulates tumor response to checkpoint blockade immunotherapy; however, this has not been well-characterized in human cancer patients. Here we examined the oral and gut microbiome of melanoma patients undergoing anti-programmed cell death 1 protein (PD-1) immunotherapy ( = 112). Significant differences were observed in the diversity and composition of the patient gut microbiome of responders versus nonresponders. Analysis of patient fecal microbiome samples ( = 43, 30 responders, 13 nonresponders) showed significantly higher alpha diversity ( < 0.01) and relative abundance of bacteria of the Ruminococcaceae family ( < 0.01) in responding patients. Metagenomic studies revealed functional differences in gut bacteria in responders, including enrichment of anabolic pathways. Immune profiling suggested enhanced systemic and antitumor immunity in responding patients with a favorable gut microbiome as well as in germ-free mice receiving fecal transplants from responding patients. Together, these data have important implications for the treatment of melanoma patients with immune checkpoint inhibitors. <<<

Junyue Cao, Malte Spielmann, Xiaojie Qiu, Xingfan Huang, Daniel M Ibrahim, Andrew J Hill, Fan Zhang, Stefan Mundlos, Lena Christiansen, Frank J Steemers, Cole Trapnell, Jay Shendure <<<

Mammalian organogenesis is a remarkable process. Within a short timeframe, the cells of the three germ layers transform into an embryo that includes most of the major internal and external organs. Here we investigate the transcriptional dynamics of mouse organogenesis at single-cell resolution. Using single-cell combinatorial indexing, we profiled the transcriptomes of around 2 million cells derived from 61 embryos staged between 9.5 and 13.5 days of gestation, in a single experiment. The resulting 'mouse organogenesis cell atlas' (MOCA) provides a global view of developmental processes during this critical window. We use Monocle 3 to identify hundreds of cell types and 56 trajectories, many of which are detected only because of the depth of cellular coverage, and collectively define thousands of corresponding marker genes. We explore the dynamics of gene expression within cell types and trajectories over time, including focused analyses of the apical ectodermal ridge, limb mesenchyme and skeletal muscle. <<<

Benoît Schmauch, Alberto Romagnoni, Elodie Pronier, Charlie Saillard, Pascale Maillé, Julien Calderaro, Aurélie Kamoun, Meriem Sefta, Sylvain Toldo, Mikhail Zaslavskiy, Thomas Clozel, Matahi Moarii, Pierre Courtiol, Gilles Wainrib <<<

Deep learning methods for digital pathology analysis are an effective way to address multiple clinical questions, from diagnosis to prediction of treatment outcomes. These methods have also been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides has yet been performed. We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone, without expert annotation. Through its interpretable design, HE2RNA provides virtual spatialization of gene expression, as validated by CD3- and CD20-staining on an independent dataset. The transcriptomic representation learned by HE2RNA can also be transferred on other datasets, even of small size, to increase prediction performance for specific molecular phenotypes. We illustrate the use of this approach in clinical diagnosis purposes such as the identification of tumors with microsatellite instability. <<<

Uncertainty estimation in medical image registration enables surgeons to evaluate the operative risk based on the trustworthiness of the registered image data thus of paramount importance for practical clinical applications. Despite the recent promising results obtained with deep unsupervised learning-based registration methods, reasoning about uncertainty of unsupervised registration models remains largely unexplored. In this work, we propose a predictive module to learn the registration and uncertainty in correspondence simultaneously. Our framework introduces empirical randomness and registration error based uncertainty prediction. We systematically assess the performances on two MRI datasets with different ensemble paradigms. Experimental results highlight that our proposed framework significantly improves the registration accuracy and uncertainty compared with the baseline. <<<

In this work, we propose a theoretical framework based on maximum profile likelihood for pairwise and groupwise registration. By an asymptotic analysis, we demonstrate that maximum profile likelihood registration minimizes an upper bound on the joint entropy of the distribution that generates the joint image data. Further, we derive the congealing method for groupwise registration by optimizing the profile likelihood in closed form, and using coordinate ascent, or iterative model refinement. We also describe a method for feature based registration in the same framework and demonstrate it on groupwise tractographic registration. In the second part of the article, we propose an approach to deep metric registration that implements maximum likelihood registration using deep discriminative classifiers. We show further that this approach can be used for maximum profile likelihood registration to discharge the need for well-registered training data, using iterative model refinement. We demonstrate that the method succeeds on a challenging registration problem where the standard mutual information approach does not perform well. <<<

PURPOSE: The accuracy of minimally invasive, intracranial neurosurgery can be challenged by deformation of brain tissue - e.g., up to 10 mm due to egress of cerebrospinal fluid during neuroendoscopic approach. We report an unsupervised, deep learning-based registration framework to resolve such deformations between preoperative MR and intraoperative CT with fast runtime for neurosurgical guidance.METHOD: The framework incorporates subnetworks for MR and CT image synthesis with a dual-channel registration subnetwork (with synthesis uncertainty providing spatially varying weights on the dual-channel loss) to estimate a diffeomorphic deformation field from both the MR and CT channels. An end-to-end training is proposed that jointly optimizes both the synthesis and registration subnetworks. The proposed framework was investigated using three datasets: (1) paired MR/CT with simulated deformations; (2) paired MR/CT with real deformations; and (3) a neurosurgery dataset with real deformation. Two state-of-the-art methods (Symmetric Normalization and VoxelMorph) were implemented as a basis of comparison, and variations in the proposed dual-channel network were investigated, including single-channel registration, fusion without uncertainty weighting, and conventional sequential training of the synthesis and registration subnetworks.RESULTS: The proposed method achieved: (1) Dice coefficient = 0.82±0.07 and TRE = 1.2 ± 0.6 mm on paired MR/CT with simulated deformations; (2) Dice coefficient = 0.83 ± 0.07 and TRE = 1.4 ± 0.7 mm on paired MR/CT with real deformations; and (3) Dice = 0.79 ± 0.13 and TRE = 1.6 ± 1.0 mm on the neurosurgery dataset with real deformations. The dual-channel registration with uncertainty weighting demonstrated superior performance (e.g., TRE = 1.2 ± 0.6 mm) compared to single-channel registration (TRE = 1.6 ± 1.0 mm, p < 0.05 for CT channel and TRE = 1.3 ± 0.7 mm for MR channel) and dual-channel registration without uncertainty weighting (TRE = 1.4 ± 0.8 mm, p < 0.05). End-to-end training of the synthesis and registration subnetworks also improved performance compared to the conventional sequential training strategy (TRE = 1.3 ± 0.6 mm). Registration runtime with the proposed network was ∼3 s.CONCLUSION: The deformable registration framework based on dual-channel MR/CT registration with spatially varying weights and end-to-end training achieved geometric accuracy and runtime that was superior to state-of-the-art baseline methods and various ablations of the proposed network. The accuracy and runtime of the method may be compatible with the requirements of high-precision neurosurgery. <<<

This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent. Extensive experiments on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in discovering high-performance convolutional architectures for image classification and recurrent architectures for language modeling, while being orders of magnitude faster than state-of-the-art non-differentiable techniques. Our implementation has been made publicly available to facilitate further research on efficient architecture search algorithms. <<<

The expanding scale and inherent complexity of biological data have encouraged a growing use of machine learning in biology to build informative and predictive models of the underlying biological processes. All machine learning techniques fit models to data; however, the specific methods are quite varied and can at first glance seem bewildering. In this Review, we aim to provide readers with a gentle introduction to a few key machine learning techniques, including the most recently developed and widely used techniques involving deep neural networks. We describe how different techniques may be suited to specific types of biological data, and also discuss some best practices and points to consider when one is embarking on experiments involving machine learning. Some emerging directions in machine learning methodology are also discussed. <<<

The translational challenges in the field of precision oncology are in part related to the biological complexity and diversity of this disease. Technological advances in genomics have facilitated large sequencing efforts and discoveries that have further supported this notion. In this review, we reflect on the impact of these discoveries on our understanding of several concepts: cancer initiation, cancer prevention, early detection, adjuvant therapy and minimal residual disease monitoring, cancer drug resistance, and cancer evolution in metastasis. We discuss key areas of focus for improving cancer outcomes, from biological insights to clinical application, and suggest where the development of these technologies will lead us. Finally, we discuss practical challenges to the wider adoption of molecular profiling in the clinic and the need for robust translational infrastructure. <<<

Rui Tao, Yanhong Wang, Yun Hu, Yaoge Jiao, Lifang Zhou, Lurong Jiang, Li Li, Xingyu He, Min Li, Yamei Yu, Qiang Chen, Shaohua Yao <<<

Large scale genomic aberrations including duplication, deletion, translocation, and other structural changes are the cause of a subtype of hereditary genetic disorders and contribute to onset or progress of cancer. The current prime editor, PE2, consisting of Cas9-nickase and reverse transcriptase enables efficient editing of genomic deletion and insertion, however, at small scale. Here, we designed a novel prime editor by fusing reverse transcriptase (RT) to nuclease wild-type Cas9 (WT-PE) to edit large genomic fragment. WT-PE system simultaneously introduced a double strand break (DSB) and a single 3' extended flap in the target site. Coupled with paired prime editing guide RNAs (pegRNAs) that have complementary sequences in their 3' terminus while target different genomic regions, WT-PE produced bi-directional prime editing, which enabled efficient and versatile large-scale genome editing, including large fragment deletion up to 16.8 megabase (Mb) pairs and chromosomal translocation. Therefore, our WT-PE system has great potential to model or treat diseases related to large-fragment aberrations. <<<

White matter hyperintensities (WMH) of presumed vascular origin are frequently found in MRIs of healthy older adults. WMH are also associated with aging and cognitive decline. Here, we compared and validated three algorithms for WMH extraction: FreeSurfer (T1w), UBO Detector (T1w + FLAIR), and FSL's Brain Intensity AbNormality Classification Algorithm (BIANCA; T1w + FLAIR) using a longitudinal dataset comprising MRI data of cognitively healthy older adults (baseline N = 231, age range 64-87 years). As reference we manually segmented WMH in T1w, three-dimensional (3D) FLAIR, and two-dimensional (2D) FLAIR images which were used to assess the segmentation accuracy of the different automated algorithms. Further, we assessed the relationships of WMH volumes provided by the algorithms with Fazekas scores and age. FreeSurfer underestimated the WMH volumes and scored worst in Dice Similarity Coefficient (DSC = 0.434) but its WMH volumes strongly correlated with the Fazekas scores (r = 0.73). BIANCA accomplished the highest DSC (0.602) in 3D FLAIR images. However, the relations with the Fazekas scores were only moderate, especially in the 2D FLAIR images (r = 0.41), and many outlier WMH volumes were detected when exploring within-person trajectories (2D FLAIR: ~30%). UBO Detector performed similarly to BIANCA in DSC with both modalities and reached the best DSC in 2D FLAIR (0.531) without requiring a tailored training dataset. In addition, it achieved very high associations with the Fazekas scores (2D FLAIR: r = 0.80). In summary, our results emphasize the importance of carefully contemplating the choice of the WMH segmentation algorithm and MR-modality. <<<

Inferring single-cell compositions and their contributions to global gene expression changes from bulk RNA sequencing (RNA-seq) datasets is a major challenge in oncology. Here we develop Bayesian cell proportion reconstruction inferred using statistical marginalization (BayesPrism), a Bayesian method to predict cellular composition and gene expression in individual cell types from bulk RNA-seq, using patient-derived, scRNA-seq as prior information. We conduct integrative analyses in primary glioblastoma, head and neck squamous cell carcinoma and skin cutaneous melanoma to correlate cell type composition with clinical outcomes across tumor types, and explore spatial heterogeneity in malignant and nonmalignant cell states. We refine current cancer subtypes using gene expression annotation after exclusion of confounding nonmalignant cells. Finally, we identify genes whose expression in malignant cells correlates with macrophage infiltration, T cells, fibroblasts and endothelial cells across multiple tumor types. Our work introduces a new lens to accurately infer cellular composition and expression in large cohorts of bulk RNA-seq data. <<<

A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and tractability. The essential idea, inspired by non-equilibrium statistical physics, is to systematically and slowly destroy structure in a data distribution through an iterative forward diffusion process. We then learn a reverse diffusion process that restores structure in data, yielding a highly flexible and tractable generative model of the data. This approach allows us to rapidly learn, sample from, and evaluate probabilities in deep generative models with thousands of layers or time steps, as well as to compute conditional and posterior probabilities under the learned model. We additionally release an open source reference implementation of the algorithm. <<<