An increased interest in longitudinal neurodevelopment during the first few years after birth has emerged in recent years. Noninvasive magnetic resonance imaging (MRI) can provide crucial information about the development of brain structures in the early months of life. Despite the success of MRI collections and analysis for adults, it remains a challenge for researchers to collect high-quality multimodal MRIs from developing infant brains because of their irregular sleep pattern, limited attention, inability to follow instructions to stay still during scanning. In addition, there are limited analytic approaches available. These challenges often lead to a significant reduction of usable MRI scans and pose a problem for modeling neurodevelopmental trajectories. Researchers have explored solving this problem by synthesizing realistic MRIs to replace corrupted ones. Among synthesis methods, the convolutional neural network-based (CNN-based) generative adversarial networks (GANs) have demonstrated promising performance. In this study, we introduced a novel 3D MRI synthesis framework- pyramid transformer network (PTNet3D)- which relies on attention mechanisms through transformer and performer layers. We conducted extensive experiments on high-resolution Developing Human Connectome Project (dHCP) and longitudinal Baby Connectome Project (BCP) datasets. Compared with CNN-based GANs, PTNet3D consistently shows superior synthesis accuracy and superior generalization on two independent, large-scale infant brain MRI datasets. Notably, we demonstrate that PTNet3D synthesized more realistic scans than CNN-based models when the input is from multi-age subjects. Potential applications of PTNet3D include synthesizing corrupted or missing images. By replacing corrupted scans with synthesized ones, we observed significant improvement in infant whole brain segmentation. <<<

We propose an effective denoising diffusion model for generatinghigh-resolution images (e.g., 1024$\times$512), trained on small-size imagepatches (e.g., 64$\times$64). We name our algorithm Patch-DM, in which a newfeature collage strategy is designed to avoid the boundary artifact whensynthesizing large-size images. Feature collage systematically crops andcombines partial features of the neighboring patches to predict the features ofa shifted image patch, allowing the seamless generation of the entire image dueto the overlap in the patch feature space. Patch-DM produces high-quality imagesynthesis results on our newly collected dataset of nature images(1024$\times$512), as well as on standard benchmarks of smaller sizes(256$\times$256), including LSUN-Bedroom, LSUN-Church, and FFHQ. We compare ourmethod with previous patch-based generation methods and achievestate-of-the-art FID scores on all four datasets. Further, Patch-DM alsoreduces memory complexity compared to the classic diffusion models. <<<

Zilong Zeng, Jiaying Zhang, Xinyuan Liang, Lianglong Sun, Yihe Zhang, Weiwei Men, Yanpei Wang, Rui Chen, Haibo Zhang, Shuping Tan, Jia-Hong Gao, Shaozheng Qin, Qiqi Tong, Hongjian He, Sha Tao, Qi Dong, Yong He, Tengda Zhao <<<

AbstractSusceptibility artifacts (SAs), which are inevitable for modern diffusion brain MR images with single-shot echo planar imaging (EPI) protocols in wide large-scale neuroimaging datasets, severely hamper the accurate detection of the human brain white matter structure. While several conventional and deep-learning based distortion correction methods have been proposed, the correction quality and model generality of these approaches are still limited. Here, we proposed the SACNet, a flexible SAs correction (SAC) framework for brain diffusion MR images of various phase-encoding EPI protocols based on an unsupervised learning-based registration convolutional neural network. This method could generate smooth diffeomorphic warps with optional neuroanatomy guidance to correct both geometric and intensity distortions of SAs. By employing near 2000 brain scans covering neonatal, child, adult and traveling participants, our SACNet consistently demonstrates state-of-the-art correction performance and effectively eliminates SAs-related multicenter effects compared with existing SAC methods. To facilitate the development of standard SAC tools for future neuroimaging studies, we also created easy-to-use command lines incorporating containerization techniques for quick user deployment. <<<

AbstractThe infant brain develops rapidly and this area of research has great clinical implications. Neurodevelopmental disorders such as autism and developmental delay have their origins, potentially, in abnormal early brain maturation. Searching for potential early neural markers requiresa prioriknowledge about infant brain development and anatomy. One of the most common methods of characterizing brain features requires normalization of individual images into a standard stereotactic space and conduct of group-based analyses in this space. A population representative brain template is critical for these population-based studies. Little research is available on constructing brain templates for typical developing Chinese infants. In the present work, a total of 112 babies from 6 to 98 days of age were included with high resolution structural magnetic resonance imaging scans. T1-weighted and T2-weighted templates were constructed using an unbiased registration approach for babies from newborn to 3 months of age. Age-specific templates were also estimated for babies aged at 0, 1, 2 and 3 months old. Then we conducted a series of evaluations and statistical analyses over whole tissue segmentations and brain parcellations. Compared to the use of population mismatched templates, using our established templates resulted in lower deformation energy to transform individual images into the template space and produced a smaller registration error, i.e., smaller standard deviation of the registered images. Significant volumetric growth was observed across total brain tissues and most of the brain regions within the first three months of age. The total brain tissues exhibited larger volumes in baby boys compared to baby girls. To the best of our knowledge, this is the first study focusing on the construction of Chinese infant brain templates. These templates can be used for investigating birth related conditions such as preterm birth, detecting neural biomarkers for neurological and neurodevelopmental disorders in Chinese populations, and exploring genetic and cultural effects on the brain. <<<

Deformable image registration is one of the fundamental tasks in medical imaging. Classical registration algorithms usually require a high computational cost for iterative optimizations. Although deep-learning-based methods have been developed for fast image registration, it is still challenging to obtain realistic continuous deformations from a moving image to a fixed image with less topological folding problem. To address this, here we present a novel diffusion-model-based image registration method, called DiffuseMorph. DiffuseMorph not only generates synthetic deformed images through reverse diffusion but also allows image registration by deformation fields. Specifically, the deformation fields are generated by the conditional score function of the deformation between the moving and fixed images, so that the registration can be performed from continuous deformation by simply scaling the latent feature of the score. Experimental results on 2D facial and 3D medical image registration tasks demonstrate that our method provides flexible deformations with topology preservation capability. <<<

We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics. Our best results are obtained by training on a weighted variational bound designed according to a novel connection between diffusion probabilistic models and denoising score matching with Langevin dynamics, and our models naturally admit a progressive lossy decompression scheme that can be interpreted as a generalization of autoregressive decoding. On the unconditional CIFAR10 dataset, we obtain an Inception score of 9.46 and a state-of-the-art FID score of 3.17. On 256x256 LSUN, we obtain sample quality similar to ProgressiveGAN. Our implementation is available at this https URL <<<

Diffusion-weighted magnetic resonance imaging (DW-MRI) is a non-invasive way of imaging white matter tracts in the human brain. DW-MRIs are usually acquired using echo-planar imaging (EPI) with high gradient fields, which could introduce severe geometric distortions that interfere with further analyses. Most tools for correcting distortion require two minimally weighted DW-MRI images (B0) acquired with different phase-encoding directions, and they can take hours to process per subject. Since a great amount of diffusion data are only acquired with a single phase-encoding direction, the application of existing approaches is limited. We propose a deep learning-based registration approach to correct distortion using only the B0 acquired from a single phase-encoding direction. Specifically, we register undistorted T1-weighted images and distorted B0 to remove the distortion through a deep learning model. We apply a differentiable mutual information loss during training to improve inter-modality alignment. Experiments on the Human Connectome Project dataset show the proposed method outperforms SyN and VoxelMorph on several metrics, and only takes a few seconds to process one subject. <<<

Background Approximately 40% of pancreatic tumors smaller than 2 cm are missed at abdominal CT. Purpose To develop and to validate a deep learning (DL)-based tool able to detect pancreatic cancer at CT. Materials and Methods Retrospectively collected contrast-enhanced CT studies in patients diagnosed with pancreatic cancer between January 2006 and July 2018 were compared with CT studies of individuals with a normal pancreas (control group) obtained between January 2004 and December 2019. An end-to-end tool comprising a segmentation convolutional neural network (CNN) and a classifier ensembling five CNNs was developed and validated in the internal test set and a nationwide real-world validation set. The sensitivities of the computer-aided detection (CAD) tool and radiologist interpretation were compared using the McNemar test. Results A total of 546 patients with pancreatic cancer (mean age, 65 years ± 12 [SD], 297 men) and 733 control subjects were randomly divided into training, validation, and test sets. In the internal test set, the DL tool achieved 89.9% (98 of 109; 95% CI: 82.7, 94.9) sensitivity and 95.9% (141 of 147; 95% CI: 91.3, 98.5) specificity (area under the receiver operating characteristic curve [AUC], 0.96; 95% CI: 0.94, 0.99), without a significant difference ( = .11) in sensitivity compared with the original radiologist report (96.1% [98 of 102]; 95% CI: 90.3, 98.9). In a test set of 1473 real-world CT studies (669 malignant, 804 control) from institutions throughout Taiwan, the DL tool distinguished between CT malignant and control studies with 89.7% (600 of 669; 95% CI: 87.1, 91.9) sensitivity and 92.8% specificity (746 of 804; 95% CI: 90.8, 94.5) (AUC, 0.95; 95% CI: 0.94, 0.96), with 74.7% (68 of 91; 95% CI: 64.5, 83.3) sensitivity for malignancies smaller than 2 cm. Conclusion The deep learning-based tool enabled accurate detection of pancreatic cancer on CT scans, with reasonable sensitivity for tumors smaller than 2 cm. © RSNA, 2022 See also the editorial by Aisen and Rodrigues in this issue. <<<

Hongxi Zhang, Jia Li, Xiaoli Su, Yang Hu, Tianmei Liu, Shaoqing Ni, Haifeng Li, Xi-Nian Zuo, Junfen Fu, Ti-Fei Yuan, Zhi Yang <<<

Brain development from 1 to 6 years of age anchors a wide range of functional capabilities and carries early signs of neurodevelopmental disorders. However, quantitative models for depicting brain morphology changes and making individualized inferences are lacking, preventing the identification of early brain atypicality during this period. With a sample size of 285, we characterized the age dependence of the cortical thickness and subcortical volume in neurologically normal children and constructed quantitative growth charts of all brain regions for preschool children. While the cortical thickness of most brain regions decreased with age, the entorhinal and parahippocampal regions displayed an inverted-U shape of age dependence. Compared to the cortical thickness, the normalized volume of subcortical regions exhibited more divergent trends, with some regions increasing, some decreasing, and some displaying inverted-U-shaped trends. The growth curve models for all brain regions demonstrated utilities in identifying brain atypicality. The percentile measures derived from the growth curves facilitate the identification of children with developmental speech and language disorders with an accuracy of 0.875 (area under the receiver operating characteristic curve: 0.943). Our results fill the knowledge gap in brain morphometrics in a critical development period and provide an avenue for individualized brain developmental status evaluation with demonstrated sensitivity. The brain growth charts are shared with the public (http://phi-group.top/resources.html). <<<

We describe a technique for simulation and correction of the effects of an arbitrary distribution of undesired components of the static and gradient magnetic fields. This technique is applicable to direct Fourier NMR imaging. The mathematical basis and details of this technique are fully described. Computer simulation demonstrates the effectiveness of this method. <<<

Brain atlases are spatial references for integrating, processing, and analyzing brain features gathered from different individuals, sources, and scales. Here we introduce a collection of joint surface-volume atlases that chart postnatal development of the human brain in a spatiotemporally dense manner from two weeks to two years of age. Our month-specific atlases chart normative patterns and capture key traits of early brain development and are therefore conducive to identifying aberrations from normal developmental trajectories. These atlases will enhance our understanding of early structural and functional development by facilitating the mapping of diverse features of the infant brain to a common reference frame for precise multifaceted quantification of cortical and subcortical changes. <<<

In neuroimaging, cortical surface atlases play a fundamental role for spatial normalization, analysis, visualization, and comparison of results across individuals and different studies. However, existing cortical surface atlases created for adults are not suitable for infant brains during the first two postnatal years, which is the most dynamic period of postnatal structural and functional development of the highly-folded cerebral cortex. Therefore, spatiotemporal cortical surface atlases for infant brains are highly desired yet still lacking for accurate mapping of early dynamic brain development. To bridge this significant gap, leveraging our infant-dedicated computational pipeline for cortical surface-based analysis and the unique longitudinal infant MRI dataset acquired in our research center, in this paper, we construct the first spatiotemporal (4D) high-definition cortical surface atlases for the dynamic developing infant cortical structures at seven time points, including 1, 3, 6, 9, 12, 18, and 24 months of age, based on 202 serial MRI scans from 35 healthy infants. For this purpose, we develop a novel method to ensure the longitudinal consistency and unbiasedness to any specific subject and age in our 4D infant cortical surface atlases. Specifically, we first compute the within-subject mean cortical folding by unbiased groupwise registration of longitudinal cortical surfaces of each infant. Then we establish longitudinally-consistent and unbiased inter-subject cortical correspondences by groupwise registration of the geometric features of within-subject mean cortical folding across all infants. Our 4D surface atlases capture both longitudinally-consistent dynamic mean shape changes and the individual variability of cortical folding during early brain development. Experimental results on two independent infant MRI datasets show that using our 4D infant cortical surface atlases as templates leads to significantly improved accuracy for spatial normalization of cortical surfaces across infant individuals, in comparison to the infant surface atlases constructed without longitudinal consistency and also the FreeSurfer adult surface atlas. Moreover, based on our 4D infant surface atlases, for the first time, we reveal the spatially-detailed, region-specific correlation patterns of the dynamic cortical developmental trajectories between different cortical regions during early brain development. <<<

This paper introduces Quicksilver, a fast deformable image registration method. Quicksilver registration for image-pairs works by patch-wise prediction of a deformation model based directly on image appearance. A deep encoder-decoder network is used as the prediction model. While the prediction strategy is general, we focus on predictions for the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed diffeomorphic mappings for sufficiently strong regularization. We also provide a probabilistic version of our prediction network which can be sampled during the testing time to calculate uncertainties in the predicted deformations. Finally, we introduce a new correction network which greatly increases the prediction accuracy of an already existing prediction network. We show experimental results for uni-modal atlas-to-image as well as uni-/multi-modal image-to-image registrations. These experiments demonstrate that our method accurately predicts registrations obtained by numerical optimization, is very fast, achieves state-of-the-art registration results on four standard validation datasets, and can jointly learn an image similarity measure. Quicksilver is freely available as an open-source software. <<<

Andreas Schuh , Antonios Makropoulos , Emma C. Robinson , Lucilio Cordero-Grande , Emer Hughes , Jana Hutter , Anthony N. Price , Maria Murgasova , Rui Pedro A. G. Teixeira , Nora Tusor , Johannes K. Steinweg , Suresh Victor , Mary A. Rutherford , Joseph V. Hajnal , A. David Edwards , Daniel Rueckert <<<

AbstractPremature birth increases the risk of developing neurocognitive and neurobe-havioural disorders. The mechanisms of altered brain development causing these disorders are yet unknown. Studying the morphology and function of the brain during maturation provides us not only with a better understanding of normal development, but may help us to identify causes of abnormal development and their consequences. A particular difficulty is to distinguish abnormal patterns of neurodevelopment from normal variation. The Developing Human Connectome Project (dHCP) seeks to create a detailed four-dimensional (4D) connectome of early life. This connectome may provide insights into normal as well as abnormal patterns of brain development. As part of this project, more than a thousand healthy fetal and neonatal brains will be scanned in vivo. This requires computational methods which scale well to larger data sets. We propose a novel groupwise method for the construction of a spatio-temporal model of mean morphology from cross-sectional brain scans at different gestational ages. This model scales linearly with the number of images and thus improves upon methods used to build existing public neonatal atlases, which derive correspondence between all pairs of images. By jointly estimating mean shape and longitudinal change, the atlas created with our method overcomes temporal inconsistencies, which are encountered when mean shape and intensity images are constructed separately for each time point. Using this approach, we have constructed a spatio-temporal atlas from 275 healthy neonates between 35 and 44 weeks post-menstrual age (PMA). The resulting atlas qualitatively preserves cortical details significantly better than publicly available atlases. This is moreover confirmed by a number of quantitative measures of the quality of the spatial normalisation and sharpness of the resulting template brain images. <<<

Atlas building and image registration are important tasks for medical image analysis. Once one or multiple atlases from an image population have been constructed, commonly (1) images are warped into an atlas space to study intra-subject or inter-subject variations or (2) a possibly probabilistic atlas is warped into image space to assign anatomical labels. Atlas estimation and nonparametric transformations are computationally expensive as they usually require numerical optimization. Additionally, previous approaches for atlas building often define similarity measures between a fuzzy atlas and each individual image, which may cause alignment difficulties because a fuzzy atlas does not exhibit clear anatomical structures in contrast to the individual images. This work explores using a convolutional neural network (CNN) to jointly predict the atlas and a stationary velocity field (SVF) parameterization for diffeomorphic image registration with respect to the atlas. Our approach does not require affine pre-registrations and utilizes pairwise image alignment losses to increase registration accuracy. We evaluate our model on 3D knee magnetic resonance images (MRI) from the OAI-ZIB dataset. Our results show that the proposed framework achieves better performance than other state-of-the-art image registration algorithms, allows for end-to-end training, and for fast inference at test time. <<<

In this work, we propose a theoretical framework based on maximum profile likelihood for pairwise and groupwise registration. By an asymptotic analysis, we demonstrate that maximum profile likelihood registration minimizes an upper bound on the joint entropy of the distribution that generates the joint image data. Further, we derive the congealing method for groupwise registration by optimizing the profile likelihood in closed form, and using coordinate ascent, or iterative model refinement. We also describe a method for feature based registration in the same framework and demonstrate it on groupwise tractographic registration. In the second part of the article, we propose an approach to deep metric registration that implements maximum likelihood registration using deep discriminative classifiers. We show further that this approach can be used for maximum profile likelihood registration to discharge the need for well-registered training data, using iterative model refinement. We demonstrate that the method succeeds on a challenging registration problem where the standard mutual information approach does not perform well. <<<

As an asset pricing model in economics and finance, factor model has been widely used in quantitative investment. Towards building more effective factor models, recent years have witnessed the paradigm shift from linear models to more flexible nonlinear data-driven machine learning models. However, due to low signal-to-noise ratio of the financial data, it is quite challenging to learn effective factor models. In this paper, we propose a novel factor model, FactorVAE, as a probabilistic model with inherent randomness for noise modeling. Essentially, our model integrates the dynamic factor model (DFM) with the variational autoencoder (VAE) in machine learning, and we propose a prior-posterior learning method based on VAE, which can effectively guide the learning of model by approximating an optimal posterior factor model with future information. Particularly, considering that risk modeling is important for the noisy stock data, FactorVAE can estimate the variances from the distribution over the latent space of VAE, in addition to predicting returns. The experiments on the real stock market data demonstrate the effectiveness of FactorVAE, which outperforms various baseline methods. <<<

The accumulation of multisite large-sample MRI datasets collected during large brain research projects in the last decade has provided critical resources for understanding the neurobiological mechanisms underlying cognitive functions and brain disorders. However, the significant site effects observed in imaging data and their derived structural and functional features have prevented the derivation of consistent findings across multiple studies. The development of harmonization methods that can effectively eliminate complex site effects while maintaining biological characteristics in neuroimaging data has become a vital and urgent requirement for multisite imaging studies. Here, we propose a deep learning-based framework to harmonize imaging data obtained from pairs of sites, in which site factors and brain features can be disentangled and encoded. We trained the proposed framework with a publicly available traveling subject dataset from the Strategic Research Program for Brain Sciences (SRPBS) and harmonized the gray matter volume maps derived from eight source sites to a target site. The proposed framework significantly eliminated intersite differences in gray matter volumes. The embedded encoders successfully captured both the abstract textures of site factors and the concrete brain features. Moreover, the proposed framework exhibited outstanding performance relative to conventional statistical harmonization methods in terms of site effect removal, data distribution homogenization, and intrasubject similarity improvement. Finally, the proposed harmonization network provided fixable expandability, through which new sites could be linked to the target site via indirect schema without retraining the whole model. Together, the proposed method offers a powerful and interpretable deep learning-based harmonization framework for multisite neuroimaging data that can enhance reliability and reproducibility in multisite studies regarding brain development and brain disorders. <<<

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to contrast introduced by magnetic resonance imaging (MRI). While classical registration methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning-based techniques are fast at test time but limited to registering images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency on training data by leveraging a generative strategy for diverse synthetic label maps and images that exposes networks to a wide range of variability, forcing them to learn more invariant features. This approach results in powerful networks that accurately generalize to a broad array of MRI contrasts. We present extensive experiments with a focus on 3D neuroimaging, showing that this strategy enables robust and accurate registration of arbitrary MRI contrasts even if the target contrast is not seen by the networks during training. We demonstrate registration accuracy surpassing the state of the art both within and across contrasts, using a single model. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images. Our code is available at doic https://w3id.org/synthmorph. <<<

Tao Zhong, Jingkuan Wei, Kunhua Wu, Liangjun Chen, Fenqiang Zhao, Yuchen Pei, Ya Wang, Hongjiang Zhang, Zhengwang Wu, Ying Huang, Tengfei Li, Li Wang, Yongchang Chen, Weizhi Ji, Yu Zhang, Gang Li, Yuyu Niu <<<

Longitudinal brain imaging atlases with densely sampled time-points and ancillary anatomical information are of fundamental importance in studying early developmental characteristics of human and non-human primate brains during infancy, which feature extremely dynamic imaging appearance, brain shape and size. However, for non-human primates, which are highly valuable animal models for understanding human brains, the existing brain atlases are mainly developed based on adults or adolescents, denoting a notable lack of temporally densely-sampled atlases covering the dynamic early brain development. To fill this critical gap, in this paper, we construct a comprehensive set of longitudinal brain atlases and associated tissue probability maps (gray matter, white matter, and cerebrospinal fluid) with totally 12 time-points from birth to 4 years of age (i.e., 1, 2, 3, 4, 5, 6, 9, 12, 18, 24, 36, and 48 months of age) based on 175 longitudinal structural MRI scans from 39 typically-developing cynomolgus macaques, by leveraging state-of-the-art computational techniques tailored for early developing brains. Furthermore, to facilitate region-based analysis using our atlases, we also provide two popular hierarchy parcellations, i.e., cortical hierarchy maps (6 levels) and subcortical hierarchy maps (6 levels), on our longitudinal macaque brain atlases. These early developing atlases, which have the densest time-points during infancy (to the best of our knowledge), will greatly facilitate the studies of macaque brain development. <<<