Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. However, most existing pre-trained models only excel in either understanding-based tasks or generation-based tasks. Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision. In this paper, we propose BLIP, a new VLP framework which transfers flexibly to both vision-language understanding and generation tasks. BLIP effectively utilizes the noisy web data by bootstrapping the captions, where a captioner generates synthetic captions and a filter removes the noisy ones. We achieve state-of-the-art results on a wide range of vision-language tasks, such as image-text retrieval (+2.7% in average recall@1), image captioning (+2.8% in CIDEr), and VQA (+1.6% in VQA score). BLIP also demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner. Code, models, and datasets are released at this https URL. <<<

Research on child development increasingly includes data on both parents and from different cultures. However, the relative importance of fathers versus mothers for child adjustment is still under debate. The present review compares the contributions of perceived paternal and maternal acceptance to various child adjustment indicators among samples of families around the world. We reviewed 127 published studies that included child-reported paternal and maternal acceptance and developmental outcomes. Regardless of the sex of parent, children benefited from perceived parental acceptance. Fathers and mothers were often found to both predict adjustment significantly to varying degrees. Paternal acceptance tended to be related to children's problem behavior and psychopathology, whereas maternal acceptance was more likely to contribute to socioemotional development. Paternal and maternal acceptance also often jointly contributed to child adjustment through their interaction with each other and with other predictors. Moreover, the link between parental acceptance and adjustment was often moderated by child gender and cultural context. <<<

Lei Zhong, Yueshan Li, Liang Xiong, Wenjing Wang, Ming Wu, Ting Yuan, Wei Yang, Chenyu Tian, Zhuang Miao, Tianqi Wang, Shengyong Yang <<<

Due to the advantages in efficacy and safety compared with traditional chemotherapy drugs, targeted therapeutic drugs have become mainstream cancer treatments. Since the first tyrosine kinase inhibitor imatinib was approved to enter the market by the US Food and Drug Administration (FDA) in 2001, an increasing number of small-molecule targeted drugs have been developed for the treatment of malignancies. By December 2020, 89 small-molecule targeted antitumor drugs have been approved by the US FDA and the National Medical Products Administration (NMPA) of China. Despite great progress, small-molecule targeted anti-cancer drugs still face many challenges, such as a low response rate and drug resistance. To better promote the development of targeted anti-cancer drugs, we conducted a comprehensive review of small-molecule targeted anti-cancer drugs according to the target classification. We present all the approved drugs as well as important drug candidates in clinical trials for each target, discuss the current challenges, and provide insights and perspectives for the research and development of anti-cancer drugs. <<<

One of the most salient features of life is its capacity to handle novelty and namely to thrive and adapt to new circumstances and changes in both the environment and internal components. An understanding of this capacity is central to several fields: the evolution of form and function, the design of effective strategies for biomedicine, and the creation of novel life forms via chimeric and bioengineering technologies. Here, we review instructive examples of living organisms solving diverse problems and propose competent navigation in arbitrary spaces as an invariant for thinking about the scaling of cognition during evolution. We argue that our innate capacity to recognize agency and intelligence in unfamiliar guises lags far behind our ability to detect it in familiar behavioral contexts. The multi-scale competency of life is essential to adaptive function, potentiating evolution and providing strategies for top-down control (not micromanagement) to address complex disease and injury. We propose an observer-focused viewpoint that is agnostic about scale and implementation, illustrating how evolution pivoted similar strategies to explore and exploit metabolic, transcriptional, morphological, and finally 3D motion spaces. By generalizing the concept of behavior, we gain novel perspectives on evolution, strategies for system-level biomedical interventions, and the construction of bioengineered intelligences. This framework is a first step toward relating to intelligence in highly unfamiliar embodiments, which will be essential for progress in artificial intelligence and regenerative medicine and for thriving in a world increasingly populated by synthetic, bio-robotic, and hybrid beings. <<<

Within computational neuroscience, the algorithmic and neural basis of structure learning remains poorly understood. Concept learning is one primary example, which requires both a type of internal model expansion process (adding novel hidden states that explain new observations), and a model reduction process (merging different states into one underlying cause and thus reducing model complexity via meta-learning). Although various algorithmic models of concept learning have been proposed within machine learning and cognitive science, many are limited to various degrees by an inability to generalize, the need for very large amounts of training data, and/or insufficiently established biological plausibility. Using concept learning as an example case, we introduce a novel approach for modeling structure learning-and specifically state-space expansion and reduction-within the active inference framework and its accompanying neural process theory. Our aim is to demonstrate its potential to facilitate a novel line of active inference research in this area. The approach we lay out is based on the idea that a generative model can be equipped with extra (hidden state or cause) "slots" that can be engaged when an agent learns about novel concepts. This can be combined with a Bayesian model reduction process, in which any concept learning-associated with these slots-can be reset in favor of a simpler model with higher model evidence. We use simulations to illustrate this model's ability to add new concepts to its state space (with relatively few observations) and increase the granularity of the concepts it currently possesses. We also simulate the predicted neural basis of these processes. We further show that it can accomplish a simple form of "one-shot" generalization to new stimuli. Although deliberately simple, these simulation results highlight ways in which active inference could offer useful resources in developing neurocomputational models of structure learning. They provide a template for how future active inference research could apply this approach to real-world structure learning problems and assess the added utility it may offer. <<<

Understanding information processing in the brain-and creating general-purpose artificial intelligence-are long-standing aspirations of scientists and engineers worldwide. The distinctive features of human intelligence are high-level cognition and control in various interactions with the world including the self, which are not defined in advance and are vary over time. The challenge of building human-like intelligent machines, as well as progress in brain science and behavioural analyses, robotics, and their associated theoretical formalisations, speaks to the importance of the world-model learning and inference. In this article, after briefly surveying the history and challenges of internal model learning and probabilistic learning, we introduce the free energy principle, which provides a useful framework within which to consider neuronal computation and probabilistic world models. Next, we showcase examples of human behaviour and cognition explained under that principle. We then describe symbol emergence in the context of probabilistic modelling, as a topic at the frontiers of cognitive robotics. Lastly, we review recent progress in creating human-like intelligence by using novel probabilistic programming languages. The striking consensus that emerges from these studies is that probabilistic descriptions of learning and inference are powerful and effective ways to create human-like artificial intelligent machines and to understand intelligence in the context of how humans interact with their world. <<<

Ao Chen, Sha Liao, Mengnan Cheng, Kailong Ma, Liang Wu, Yiwei Lai, Xiaojie Qiu, Jin Yang, Jiangshan Xu, Shijie Hao, Xin Wang, Huifang Lu, Xi Chen, Xing Liu, Xin Huang, Zhao Li, Yan Hong, Yujia Jiang, Jian Peng, Shuai Liu, Mengzhe Shen, Chuanyu Liu, Quanshui Li, Yue Yuan, Xiaoyu Wei, Huiwen Zheng, Weimin Feng, Zhifeng Wang, Yang Liu, Zhaohui Wang, Yunzhi Yang, Haitao Xiang, Lei Han, Baoming Qin, Pengcheng Guo, Guangyao Lai, Pura Muñoz-Cánoves, Patrick H Maxwell, Jean Paul Thiery, Qing-Feng Wu, Fuxiang Zhao, Bichao Chen, Mei Li, Xi Dai, Shuai Wang, Haoyan Kuang, Junhou Hui, Liqun Wang, Ji-Feng Fei, Ou Wang, Xiaofeng Wei, Haorong Lu, Bo Wang, Shiping Liu, Ying Gu, Ming Ni, Wenwei Zhang, Feng Mu, Ye Yin, Huanming Yang, Michael Lisby, Richard J Cornall, Jan Mulder, Mathias Uhlén, Miguel A Esteban, Yuxiang Li, Longqi Liu, Xun Xu, Jian Wang <<<

Spatially resolved transcriptomic technologies are promising tools to study complex biological processes such as mammalian embryogenesis. However, the imbalance between resolution, gene capture, and field of view of current methodologies precludes their systematic application to analyze relatively large and three-dimensional mid- and late-gestation embryos. Here, we combined DNA nanoball (DNB)-patterned arrays and in situ RNA capture to create spatial enhanced resolution omics-sequencing (Stereo-seq). We applied Stereo-seq to generate the mouse organogenesis spatiotemporal transcriptomic atlas (MOSTA), which maps with single-cell resolution and high sensitivity the kinetics and directionality of transcriptional variation during mouse organogenesis. We used this information to gain insight into the molecular basis of spatial cell heterogeneity and cell fate specification in developing tissues such as the dorsal midbrain. Our panoramic atlas will facilitate in-depth investigation of longstanding questions concerning normal and abnormal mammalian development. <<<

William J. Sutherland , Robert P. Freckleton , H. Charles J. Godfray , Steven R. Beissinger , Tim Benton , Duncan D. Cameron , Yohay Carmel , David A. Coomes , Tim Coulson , Mark C. Emmerson , Rosemary S. Hails , Graeme C. Hays , Dave J. Hodgson , Michael J. Hutchings , David Johnson , Julia P. G. Jones , Matt J. Keeling , Hanna Kokko , William E. Kunin , Xavier Lambin , Owen T. Lewis , Yadvinder Malhi , Nova Mieszkowska , E. J. Milner-Gulland , Ken Norris , Albert B. Phillimore , Drew W. Purves , Jane M. Reid , Daniel C. Reuman , Ken Thompson , Justin M. J. Travis , Lindsay A. Turnbull , David A. Wardle , Thorsten Wiegand <<<

Fundamental ecological research is both intrinsically interesting and provides the basic knowledge required to answer applied questions of importance to the management of the natural world. The 100th anniversary of the British Ecological Society in 2013 is an opportune moment to reflect on the current status of ecology as a science and look forward to high-light priorities for future work. To do this, we identified 100 important questions of fundamental importance in pure ecology. We elicited questions from ecologists working across a wide range of systems and disciplines. The 754 questions submitted (listed in the online appendix) from 388 participants were narrowed down to the final 100 through a process of discussion, rewording and repeated rounds of voting. This was done during a two-day workshop and thereafter. The questions reflect many of the important current conceptual and technical pre-occupations of ecology. For example, many questions concerned the dynamics of environmental change and complex ecosystem interactions, as well as the interaction between ecology and evolution. The questions reveal a dynamic science with novel subfields emerging. For example, a group of questions was dedicated to disease and micro-organisms and another on human impacts and global change reflecting the emergence of new subdisciplines that would not have been foreseen a few decades ago. The list also contained a number of questions that have perplexed ecologists for decades and are still seen as crucial to answer, such as the link between population dynamics and life-history evolution. Synthesis. These 100 questions identified reflect the state of ecology today. Using them as an agenda for further research would lead to a substantial enhancement in understanding of the discipline, with practical relevance for the conservation of biodiversity and ecosystem function. <<<

Bálint Ármin Pataki, Alex Olar, Dezső Ribli, Adrián Pesti, Endre Kontsek, Benedek Gyöngyösi, Ágnes Bilecz, Tekla Kovács, Kristóf Attila Kovács, Zsófia Kramer, András Kiss, Miklós Szócska, Péter Pollner, István Csabai <<<

Histopathology is the gold standard method for staging and grading human tumors and provides critical information for the oncoteam's decision making. Highly-trained pathologists are needed for careful microscopic analysis of the slides produced from tissue taken from biopsy. This is a time-consuming process. A reliable decision support system would assist healthcare systems that often suffer from a shortage of pathologists. Recent advances in digital pathology allow for high-resolution digitalization of pathological slides. Digital slide scanners combined with modern computer vision models, such as convolutional neural networks, can help pathologists in their everyday work, resulting in shortened diagnosis times. In this study, 200 digital whole-slide images are published which were collected via hematoxylin-eosin stained colorectal biopsy. Alongside the whole-slide images, detailed region level annotations are also provided for ten relevant pathological classes. The 200 digital slides, after pre-processing, resulted in 101,389 patches. A single patch is a 512 × 512 pixel image, covering 248 × 248 μm tissue area. Versions at higher resolution are available as well. Hopefully, HunCRC, this widely accessible dataset will aid future colorectal cancer computer-aided diagnosis and research. <<<

Cristina Perez-Becerril, Andrew J Wallace, Helene Schlecht, Naomi L Bowers, Philip T Smith, Carolyn Gokhale, Helen Eaton, Chris Charlton, Rachel Robinson, Ruth S Charlton, D Gareth Evans, Miriam J Smith <<<

Schwannomatosis comprises a group of hereditary tumor predisposition syndromes characterized by, usually benign, multiple nerve sheath tumors, which frequently cause severe pain that does not typically respond to drug treatments. The most common schwannomatosis-associated gene is NF2, but SMARCB1 and LZTR1 are also associated. There are still many cases in which no pathogenic variants (PVs) have been identified, suggesting the existence of as yet unidentified genetic risk factors. In this study, we performed extended genetic screening of 75 unrelated schwannomatosis patients without identified germline PVs in NF2, LZTR1, or SMARCB1. Screening of the coding region of DGCR8, COQ6, CDKN2A, and CDKN2B was carried out, based on previous reports that point to these genes as potential candidate genes for schwannomatosis. Deletions or duplications in CDKN2A, CDKN2B, and adjacent chromosome 9 region were assessed by multiplex ligation-dependent probe amplification analysis. Sequencing analysis of a patient with multiple schwannomas and melanomas identified a novel duplication in the coding region of CDKN2A, disrupting both p14ARF and p16INK4a. Our results suggest that none of these genes are major contributors to schwannomatosis risk but the possibility remains that they may have a role in more complex mechanisms for tumor predisposition. <<<

Omics-based biomedical learning frequently relies on data of high-dimensions (up to thousands) and low-sample sizes (dozens to hundreds), which challenges efficient deep learning (DL) algorithms, particularly for low-sample omics investigations. Here, an unsupervised novel feature aggregation tool AggMap was developed to Aggregate and Map omics features into multi-channel 2D spatial-correlated image-like feature maps (Fmaps) based on their intrinsic correlations. AggMap exhibits strong feature reconstruction capabilities on a randomized benchmark dataset, outperforming existing methods. With AggMap multi-channel Fmaps as inputs, newly-developed multi-channel DL AggMapNet models outperformed the state-of-the-art machine learning models on 18 low-sample omics benchmark tasks. AggMapNet exhibited better robustness in learning noisy data and disease classification. The AggMapNet explainable module Simply-explainer identified key metabolites and proteins for COVID-19 detections and severity predictions. The unsupervised AggMap algorithm of good feature restructuring abilities combined with supervised explainable AggMapNet architecture establish a pipeline for enhanced learning and interpretability of low-sample omics data. <<<

Assaf Magen , Pauline Hamon , Nathalie Fiaschi , Leanna Troncoso , Etienne Humblin , Darwin D'souza , Travis Dawson , Matthew D. Park , Joel Kim , Steven Hamel , Mark Buckup , Christie Chang , Alexandra Tabachnikova , Hara Schwartz , Nausicaa Malissen , Yonit Lavin , Alessandra Soares-Schanoski , Bruno Giotti , Samarth Hegde , Raphael Mattiuz , Clotilde Hennequin , Jessica Le Berichel , Zhen Zhao , Stephen Ward , Isabel Fiel , Colles Price , Nicolas Fernandez , Jiang He , Baijun Kou , Michael Dobosz , Lianjie Li , Christina Adler , Min Ni , Yi Wei , Wei Wang , Namita T. Gupta , Kunal Kundu , Kamil Cygan , Raquel P. Deering , Alex Tsankov , Seunghee Kim-Schulze , Sacha Gnjatic , Ephraim Kenigsberg , Myron Schwartz , Thomas U. Marron , Gavin Thurston , Alice O. Kamphorst , Miriam Merad <<<

Here, we leveraged a large neoadjuvant PD-1 blockade trial in patients with hepatocellular carcinoma (HCC) to search for correlates of response to immune checkpoint blockade (ICB) within T cell-rich tumors. We show that ICB response correlated with the clonal expansion of intratumoral CXCL13+ CH25H+ IL-21+ PD-1+ CD4 T helper cells (CXCL13+ Th) and Granzyme K+ PD-1+ effector-like CD8 T cells, whereas terminally exhausted CD39hi TOXhi PD-1hi CD8 T cells dominated in non-responders. Strikingly, most T cell receptor (TCR) clones that expanded post-treatment were found in pre-treatment biopsies. Notably, PD-1+ TCF-1+ progenitor-like CD8 T cells were present in tumors of responders and non-responders and shared clones mainly with effector-like cells in responders or terminally differentiated cells in non-responders, suggesting that local CD8 T cell differentiation occurs upon ICB. We found that these progenitor CD8 T cells interact with CXCL13+ Th cells within cellular triads around dendritic cells enriched in maturation and regulatory molecules, or "mregDC". Receptor-ligand analysis revealed unique interactions within these triads that may promote the differentiation of progenitor CD8 T cells into effector-like cells upon ICB. These results suggest that discrete intratumoral niches that include mregDC and CXCL13+ Th cells control the differentiation of tumor-specific progenitor CD8 T cell clones in patients treated with ICB. <<<

Integration of the neural processes in the human brain is realized through interconnections that exist between different neural centers. These interconnections take place through white matter pathways. White matter tractography is currently the only available technique for the reconstruction of the anatomical connectivity in the human brain noninvasively and in vivo. The trajectory and terminations of white matter pathways are estimated from local orientations of nerve bundles. These orientations are obtained using measurements of water diffusion in the brain. In this article, the techniques for estimating fiber directions from diffusion measurements in the human brain are reviewed. Methods of white matter tractography are described, together with the current limitations of the technique, including sensitivity to image noise and partial voluming. The applications of white matter tractography to the topographical characterization of the white matter connections and the segmentation of specific white matter pathways, and corresponding functional units of gray matter, are discussed. In this context, the potential impact of white matter tractography in mapping the functional systems and subsystems in the human brain, and their interrelations, is described. Finally, the applications of white matter tractography to the study of brain disorders, including fiber tract localization in brains affected by tumors and the identification of impaired connectivity routes in neurologic and neuropsychiatric diseases, are discussed. <<<

Lohith G Kini, John M Bernabei, Fadi Mikhail, Peter Hadar, Preya Shah, Ankit N Khambhati, Kelly Oechsel, Ryan Archer, Jacqueline Boccanfuso, Erin Conrad, Russell T Shinohara, Joel M Stein, Sandhitsu Das, Ammar Kheder, Timothy H Lucas, Kathryn A Davis, Danielle S Bassett, Brian Litt <<<

Patients with drug-resistant epilepsy often require surgery to become seizure-free. While laser ablation and implantable stimulation devices have lowered the morbidity of these procedures, seizure-free rates have not dramatically improved, particularly for patients without focal lesions. This is in part because it is often unclear where to intervene in these cases. To address this clinical need, several research groups have published methods to map epileptic networks but applying them to improve patient care remains a challenge. In this study we advance clinical translation of these methods by: (i) presenting and sharing a robust pipeline to rigorously quantify the boundaries of the resection zone and determining which intracranial EEG electrodes lie within it; (ii) validating a brain network model on a retrospective cohort of 28 patients with drug-resistant epilepsy implanted with intracranial electrodes prior to surgical resection; and (iii) sharing all neuroimaging, annotated electrophysiology, and clinical metadata to facilitate future collaboration. Our network methods accurately forecast whether patients are likely to benefit from surgical intervention based on synchronizability of intracranial EEG (area under the receiver operating characteristic curve of 0.89) and provide novel information that traditional electrographic features do not. We further report that removing synchronizing brain regions is associated with improved clinical outcome, and postulate that sparing desynchronizing regions may further be beneficial. Our findings suggest that data-driven network-based methods can identify patients likely to benefit from resective or ablative therapy, and perhaps prevent invasive interventions in those unlikely to do so. <<<

Deep learning (DL) is being successfully applied across multiple domains, yet these models learn in a most artificial way: they require large quantities of labeled data to grasp even simple concepts. Thus, the main bottleneck is often access to supervised data. Here, we highlight a trend in a potential solution to this challenge: synthetic data. Synthetic data are becoming accessible due to progress in rendering pipelines, generative adversarial models, and fusion models. Moreover, advancements in domain adaptation techniques help close the statistical gap between synthetic and real data. Paradoxically, this artificial solution is also likely to enable more natural learning, as seen in biological systems, including continual, multimodal, and embodied learning. Complementary to this, simulators and deep neural networks (DNNs) will also have a critical role in providing insight into the cognitive and neural functioning of biological systems. We also review the strengths of, and opportunities and novel challenges associated with, synthetic data. <<<

CNN-based image processing has been actively applied to histopathological analysis to detect and classify cancerous tumors automatically. However, CNN-based classifiers generally predict a label with overconfidence, which becomes a serious problem in the medical domain. The objective of this study is to propose a new training method, called MixPatch, designed to improve a CNN-based classifier by specifically addressing the prediction uncertainty problem and examine its effectiveness in improving diagnosis performance in the context of histopathological image analysis. MixPatch generates and uses a new sub-training dataset, which consists of mixed-patches and their predefined ground-truth labels, for every single mini-batch. Mixed-patches are generated using a small size of clean patches confirmed by pathologists while their ground-truth labels are defined using a proportion-based soft labeling method. Our results obtained using a large histopathological image dataset shows that the proposed method performs better and alleviates overconfidence more effectively than any other method examined in the study. More specifically, our model showed 97.06% accuracy, an increase of 1.6% to 12.18%, while achieving 0.76% of expected calibration error, a decrease of 0.6% to 6.3%, over the other models. By specifically considering the mixed-region variation characteristics of histopathology images, MixPatch augments the extant mixed image methods for medical image analysis in which prediction uncertainty is a crucial issue. The proposed method provides a new way to systematically alleviate the overconfidence problem of CNN-based classifiers and improve their prediction accuracy, contributing toward more calibrated and reliable histopathology image analysis. <<<

Short-read RNA sequencing and long-read RNA sequencing each have their strengths and weaknesses for transcriptome assembly. While short reads are highly accurate, they are rarely able to span multiple exons. Long-read technology can capture full-length transcripts, but its relatively high error rate often leads to mis-identified splice sites. Here we present a new release of StringTie that performs hybrid-read assembly. By taking advantage of the strengths of both long and short reads, hybrid-read assembly with StringTie is more accurate than long-read only or short-read only assembly, and on some datasets it can more than double the number of correctly assembled transcripts, while obtaining substantially higher precision than the long-read data assembly alone. Here we demonstrate the improved accuracy on simulated data and real data from Arabidopsis thaliana, Mus musculus, and human. We also show that hybrid-read assembly is more accurate than correcting long reads prior to assembly while also being substantially faster. StringTie is freely available as open source software at https://github.com/gpertea/stringtie. <<<

The volume of data is growing exponentially and becoming more valuable to organizations that collect it, from e-commerce data, shipping, audio and video logs, text messages, internet search queries, stock market activity, financial transactions, the Internet of Things, and various other sources. The major challenges are related with the way to extract insights from such a rich data environment and whether Deep Learning can be successful with Big Data. To get some insight on these topics, social network data are employed as a case study on how sentiments can affect decisions in stock market environments. In this paper, we propose a generalized Deep Learning-based classification framework for Stock Market Sentiment Analysis. This work comprises the study, the development, and implementation of an automatic classification system based on Deep Learning and the validation of its adequacy and efficiency in any scenario, particularly Stock Market Sentiment Analysis. Distinct datasets and several Deep Learning approaches with different layers and embedded techniques are used, and their performances are evaluated. These developments show how Deep Learning reacts to distinct contexts. The results also give context on how different techniques with different parameter combinations react to certain types of data. Convolution obtained the best results when dealing with complex data inputs, and long short-term layers kept a memory of data, allowing inputs which are not as common to still be considered for decisions. The models that resulted from Stock Market Sentiment Analysis datasets were applied with some success to real-life problems. The best models reached accuracies of 73% in training and 69% in certain test datasets. In a simulation, a model was able to provide a Return on Investment of 4.4%. The results contribute to understanding how to process Big Data efficiently using Deep Learning and specialized hardware techniques. <<<

Rapid progress of industrial development, urbanization and traffic has caused air quality reduction that negatively affects human health and environmental sustainability, especially among developed countries. Numerous studies on the development of air quality forecasting model using machine learning have been conducted to control air pollution. As such, there are significant numbers of reviews on the application of machine learning in air quality forecasting. Shallow architectures of machine learning exhibit several limitations and yield lower forecasting accuracy than deep learning architecture. Deep learning is a new technology in computational intelligence; thus, its application in air quality forecasting is still limited. This study aims to investigate the deep learning applications in time series air quality forecasting. Owing to this, literature search is conducted thoroughly from all scientific databases to avoid unnecessary clutter. This study summarizes and discusses different types of deep learning algorithms applied in air quality forecasting, including the theoretical backgrounds, hyperparameters, applications and limitations. Hybrid deep learning with data decomposition, optimization algorithm and spatiotemporal models are also presented to highlight those techniques' effectiveness in tackling the drawbacks of individual deep learning models. It is clearly stated that hybrid deep learning was able to forecast future air quality with higher accuracy than individual models. At the end of the study, some possible research directions are suggested for future model development. The main objective of this review study is to provide a comprehensive literature summary of deep learning applications in time series air quality forecasting that may benefit interested researchers for subsequent research. <<<