AutoML.org

Freiburg-Hannover-Tübingen

Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

4221 entries « ‹ 51 of 85 › »

2022

1721.

Yu, Jialiang; Gao, Song; Tian, Jie; Bian, Hongli; Liu, Hui; Li, Junqing

Evolutionary Neural Architecture Search Based on Variational Inference Bayesian Convolutional Neural Network Proceedings Article

In: 2022 4th International Conference on Data-driven Optimization of Complex Systems (DOCS), pp. 1-6, 2022.

1720.

Li, Jiaquan; Hong, Haokai; Shi, Minghui; Lin, Qiuzhen; Zhou, Fenfen; Tan, Kay Chen; Jiang, Min

Knowledge transfer for Object Detection with Evolution architecture search Proceedings Article

In: 2022 4th International Conference on Data-driven Optimization of Complex Systems (DOCS), pp. 1-6, 2022.

1719.

Yuan, Jinliang; Xu, Mengwei; Zhao, Yuxin; Bian, Kaigui; Huang, Gang; Liu, Xuanzhe; Wang, Shangguang

Resource-Aware Federated Neural Architecture Search over Heterogeneous Mobile Devices Journal Article

In: IEEE Transactions on Big Data, pp. 1-11, 2022.

1718.

Zhu, Yichen; Fu, Xiaowei

BNNAS++: Towards Unbiased Neural Architecture Search With Batch Normalization Journal Article

In: IEEE Access, vol. 10, pp. 128424-128432, 2022.

1717.

Bouzidi, Halima; Odema, Mohanad; Ouarnoughi, Hamza; Faruque, Mohammad Abdullah Al; Niar, Smail

HADAS: Hardware-Aware Dynamic Neural Architecture Search for Edge Performance Scaling Technical Report

2022.

1716.

Guan, Ziyi; Zhou, Wenyong; Ren, Yuan; Xie, Rui; Yu, Hao; Wong, Ngai

A Hardware-Aware Neural Architecture Search Pareto Front Exploration for In-Memory Computing Proceedings Article

In: 2022 IEEE 16th International Conference on Solid-State & Integrated Circuit Technology (ICSICT), pp. 1-4, 2022.

1715.

Peng, Yameng; Song, Andy; Ciesielski, Vic; Fayek, Haytham M.; Chang, Xiaojun

PRE-NAS: Evolutionary Neural Architecture Search with Predictor Journal Article

In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2022.

1714.

Liu, Shiqing; Zhang, Haoyu; Jin, Yaochu

A survey on computationally efficient neural architecture search Journal Article

In: Journal of Automation and Intelligence, vol. 1, no. 1, pp. 100002, 2022, ISSN: 2949-8554.

Abstract | Links | BibTeX

1713.

Falanti, Andrea; Lomurno, Eugenio; Ardagna, Danilo; Matteucci, Matteo

POPNASv3: a Pareto-Optimal Neural Architecture Search Solution for Image and Time Series Classification Technical Report

2022.

1712.

Lee, Bokyeung; Ko, Kyungdeuk; Hong, Jonghwan; Ko, Hanseok

Single Cell Training on Architecture Search for Image Denoising Technical Report

2022.

1711.

Patel, Ria; Rorabaugh, Ariel Keller; Olaya, Paula; Caino-Lores, Silvina; Channing, Georgia; Schuman, Catherine; Miyashita, Osamu; Tama, Florence; Taufer, Michela

A Methodology to Generate Efficient Neural Networks for Classification of Scientific Datasets Proceedings Article

In: 2022 IEEE 18th International Conference on e-Science (e-Science), pp. 389-390, 2022.

1710.

Dai, Haixing; Li, Qing; Zhao, Lin; Pan, Liming; Shi, Cheng; Liu, Zhengliang; Wu, Zihao; Zhang, Lu; Zhao, Shijie; Wu, Xia; Liu, Tianming; Zhu, Dajiang

Graph Representation Neural Architecture Search for Optimal Spatial/Temporal Functional Brain Network Decomposition Proceedings Article

In: Lian, Chunfeng; Cao, Xiaohuan; Rekik, Islem; Xu, Xuanang; Cui, Zhiming (Ed.): Machine Learning in Medical Imaging, pp. 279–287, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-21014-3.

Abstract | Links | BibTeX

@inproceedings{10.1007/978-3-031-21014-3_29,

title = {Graph Representation Neural Architecture Search for Optimal Spatial/Temporal Functional Brain Network Decomposition},

author = {Haixing Dai and Qing Li and Lin Zhao and Liming Pan and Cheng Shi and Zhengliang Liu and Zihao Wu and Lu Zhang and Shijie Zhao and Xia Wu and Tianming Liu and Dajiang Zhu},

editor = {Chunfeng Lian and Xiaohuan Cao and Islem Rekik and Xuanang Xu and Zhiming Cui},

url = {https://link.springer.com/chapter/10.1007/978-3-031-21014-3_29},

isbn = {978-3-031-21014-3},

year  = {2022},

date = {2022-01-01},

urldate = {2022-01-01},

booktitle = {Machine Learning in Medical Imaging},

pages = {279--287},

publisher = {Springer Nature Switzerland},

address = {Cham},

abstract = {Decomposing the spatial/temporal functional brain networks from 4D functional magnetic resonance imaging (fMRI) data has attracted extensive attention. Among all these efforts, deep neural network-based methods have shown significant advantages due to their powerful hierarchical representation ability. However, the network architectures of those deep learning models are manually crafted, which is time consuming and non-optimal. This paper presents a novel graph representation neural architecture search (GR-NAS) method based on graph representation to optimize the vanilla RNN cell structure for decomposing spatial/temporal brain networks. The core idea is to embed the discrete search space of the RNN cell into a continuous domain that preserves the topological information. After that, popular search algorithms, e.g., reinforcement learning (RL) and Bayesian optimization (BO), can be employed to find the optimal architecture in this continuous space. The proposed method was evaluated on the Human Connectome Project (HCP) task fMRI datasets. Extensive experiments demonstrated the superiority of the proposed model in brain network decomposition both spatially and temporally. To our best knowledge, the proposed model is among the early efforts using NAS strategy to optimally decompose spatial/temporal functional brain networks from fMRI data.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

1709.

Garavagno, Andrea Mattia; Leonardis, Daniele; Frisoli, Antonio

Colab NAS: Obtaining lightweight task-specific convolutional neural networks following Occam's razor Technical Report

2022.

1708.

Shavit, Hadar; Jatelnicki, Filip; Mor-Puigventós, Pol; Kowalczyk, Wojtek

From Xception to NEXcepTion: New Design Decisions and Neural Architecture Search Technical Report

2022.

1707.

Huang, Han; Shen, Li; He, Chaoyang; Dong, Weisheng; Liu, Wei

Differentiable Neural Architecture Search for Extremely Lightweight Image Super-Resolution Journal Article

In: IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-1, 2022.

1706.

Park, Seongmin; Kwon, Beomseok; Lim, Jieun; Sim, Kyuyoung; Kim, Taeho; Choi, Jungwook

Automatic Network Adaptation for Ultra-Low Uniform-Precision Quantization Miscellaneous

2022.

1705.

Nikulenkov, Mikhail; Khamitov, Kamil; Popova, Nina

Regularization Approach for Accelerating Neural Architecture Search Proceedings Article

In: Voevodin, Vladimir; Sobolev, Sergey; Yakobovskiy, Mikhail; Shagaliev, Rashit (Ed.): Supercomputing, pp. 475–485, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-22941-1.

Abstract | Links | BibTeX

@inproceedings{10.1007/978-3-031-22941-1_35,

title = {Regularization Approach for Accelerating Neural Architecture Search},

author = {Mikhail Nikulenkov and Kamil Khamitov and Nina Popova},

editor = {Vladimir Voevodin and Sergey Sobolev and Mikhail Yakobovskiy and Rashit Shagaliev},

url = {https://link.springer.com/chapter/10.1007/978-3-031-22941-1_35},

isbn = {978-3-031-22941-1},

year  = {2022},

date = {2022-01-01},

urldate = {2022-01-01},

booktitle = {Supercomputing},

pages = {475--485},

publisher = {Springer International Publishing},

address = {Cham},

abstract = {Modern Artificial Neural networks utilizes vast topologies to become applicable to the vast majority of the ML problems. Such topologies have an enormous number of parameters nowadays, which make it easier to use, but harder to train and modify. With such a number of parameters, the usage of semi-automatic techniques for constructing or adjusting new models for topical problems is vital for the whole industry. It makes it essential to optimize such methods like Neural Architecture Search (NAS) to efficiently utilize computational resources of the GPU and whole cluster. Since modern NAS tools are commonly used to optimize models in different areas or make a combination of models from the past, they have to use a plenty of computational power to perform certain hyperparameters optimization routines. Many NAS methods are of a highly parallel nature, but still they need a lot of computational power and have an enormous convergence time. The key to increasing performance of many NAS methods is boosting performance of the training of the highly-depth and synthesized model, to evaluate the probe of the model and obtain the score for this epoch. It means that demands for parallel implementations need to be available in different cluster configurations and should utilize as many nodes as possible, showing high scalability. However, straightforward approaches where NAS solving does not consider previous results lead to wasteful utilization of computation power. In this article, we introduce a new method that can improve convergence in NEAT-based NAS method using L1/L2 regularization during the evolution step.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

1704.

Liu, Yuqiao; Li, Haipeng; Sun, Yanan; Liu, Shuaicheng

DAS: Neural Architecture Search via Distinguishing Activation Score Miscellaneous

2022.

1703.

Damarla, Anupama; Doraikannan, Sumathi

Optimized one-shot neural architecture search for skin cancer classification Journal Article

In: Journal of Electronic Imaging, vol. 31, no. 6, pp. 063053, 2022.

1702.

Wang, Zhaolei; Lu, Kunfeng; Yu, Chunmei; Yao, Na; Wang, Ludi; Zhao, Jikang

Rocket Self-learning Control based on Lightweight Neural Network Architecture Search Proceedings Article

In: 2022 IEEE International Conference on Unmanned Systems (ICUS), pp. 1621-1626, 2022.

1701.

Ou, Yuwei; Xie, Xiangning; Gao, Shangce; Sun, Yanan; Tan, Kay Chen; Lv, Jiancheng

Differentiable Search of Accurate and Robust Architectures Miscellaneous

2022.

1700.

Yao, Siya; Zhao, Yu; Deng, Qi; Ma, Jun; Kang, Qi

Multi-objective Neural Architecture Adaptation in Transfer Learning Proceedings Article

In: 2022 IEEE International Conference on Networking, Sensing and Control (ICNSC), pp. 1-5, 2022.

1699.

Yao, Siya; Zhao, Yu; Deng, Qi; Ma, Jun; Kang, Qi

Multi-objective Neural Architecture Adaptation in Transfer Learning Proceedings Article

In: 2022 IEEE International Conference on Networking, Sensing and Control (ICNSC), pp. 1-5, 2022.

1698.

Yao, Siya; Zhao, Yu; Deng, Qi; Ma, Jun; Kang, Qi

Multi-objective Neural Architecture Adaptation in Transfer Learning Proceedings Article

In: 2022 IEEE International Conference on Networking, Sensing and Control (ICNSC), pp. 1-5, 2022.

1697.

Li, Yutong

Forest fire smoke recognition and detection based on EfficientNet Proceedings Article

In: 2022 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS), pp. 712-719, 2022.

1696.

Lyu, Xinqi; Chen, Yibo; Chen, Jiamin; Liu, Xiangyue; Gao, Jianliang

Spatio-Temporal Based Architecture Topology Search for Multivariate Time Series Prediction Proceedings Article

In: 2022 IEEE International Conference on Big Data (Big Data), pp. 1304-1309, 2022.

1695.

Wei, Zimian; Pan, Hengyue; Li, Lujun; Lu, Menglong; Niu, Xin; Dong, Peijie; Li, Dongsheng

Multi-trial Neural Architecture Search with Lottery Tickets Technical Report

2022.

1694.

Xu, Binke; Bi, Ying; Xue, Bing; Schindler, Jan; Martin, Brent; Zhang, Mengjie

Automatically Designing U-Nets Using A Genetic Algorithm for Tree Image Segmentation Proceedings Article

In: 2022 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 626-633, 2022.

1693.

Zheng, Xin; Zhang, Miao; Chen, Chunyang; Li, Chaojie; Zhou, Chuan; Pan, Shirui

Multi-Relational Graph Neural Architecture Search with Fine-grained Message Passing Proceedings Article

In: 2022 IEEE International Conference on Data Mining (ICDM), pp. 783-792, 2022.

1692.

Noce, Julia; Chantong, Gianella; Jauregui, Gustavo; Mogami, Roberto; Monteiro, Alexandra; Figueiredo, Karla; Vellasco, Marley

Äpplied Enhanced Q-NAS for COVID-19 Detection in CT Images Proceedings Article

In: Mahmud, Mufti; Ieracitano, Cosimo; Kaiser, M. Shamim; Mammone, Nadia; Morabito, Francesco Carlo (Ed.): Äpplied Intelligence and Informatics", pp. 419–433, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-24801-6.

Abstract | BibTeX

1691.

Song, Li; Bao, Kainan; Ke, Songyu; Li, Chunyang; Zhang, Junbo; Zheng, Yu

Efficient Spatio-Temporal Randomly Wired Neural Networks for Traffic Forecasting Proceedings Article

In: 2022 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 1-8, 2022.

1690.

Sun, Hanbo; Wang, Chenyu; Zhu, Zhenhua; Ning, Xuefei; Dai, Guohao; Yang, Huazhong; Wang, Yu

Gibbon: Efficient Co-Exploration of NN Model and Processing-In-Memory Architecture Proceedings Article

In: 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 867-872, 2022.

1689.

Krishnakumar, Arjun; White, Colin; Zela, Arber; Tu, Renbo; Safari, Mahmoud; Hutter, Frank

NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies Proceedings Article

In: 2022.

1688.

Ghosh, Arjun; Jana, Nanda Dulal

Artificial Bee Colony Optimization based Optimal Convolutional Neural Network Architecture Design Proceedings Article

In: 2022 IEEE 19th India Council International Conference (INDICON), pp. 1-7, 2022.

2021

1687.

Jixiang Li Shun Lu, Jianchao Tan

TNASP: A Transformer-based NAS Predictor with a Self-evolution Framework Proceedings Article

In: 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021.

1686.

Hao, Jie; Zhu, William

Layered Feature Representation for Differentiable Architecture Search Technical Report

2021.

1685.

Choe, Hyeokjun; Na, Byunggook; Mok, Jisoo; Yoon, Sungroh

Variance-stationary Differentiable NAS Proceedings Article

In: British Machine Vision Conference (BMVC) 2021, 2021.

1684.

Mo, Shentong; Xia, Jingfei; Ren, Pinxu

Simulated Annealing for Neural Architecture Search Proceedings Article

In: OPT2021: 13th Annual Workshop on Optimization for Machine Learning, 2021.

1683.

Wang, Zifu; Blaschko, Matthew B.

MRF-NAS: Improving UNet with Neural Architecture Search for Medical Image Segmentation Technical Report

2021.

1682.

Huang, Ziyan; Wang, Zehua; Yang, Zhikai; Gu, Lixu

AdwU-Net: Adaptive Depth and Width U-Net for Medical Image Segmentation by Differentiable Neural Architecture Search Technical Report

2021.

1681.

Tran, Linh Tam; Baea, Sung-Ho

Training-Free Hardware-Aware Neural Architecture Search with Reinforcement Learning Technical Report

2021.

1680.

Gao, Yang; Zhang, Peng; Li, Zhao; Zhou, Chuan; Liu, Yongchao; Hu, Yue

Heterogeneous Graph Neural Architecture Search Proceedings Article

In: 2021 IEEE International Conference on Data Mining (ICDM), 2021.

1679.

Prashanth, Thota Guna Durga; Reddy, Pendli Rishith

Anomaly based Intrusion Detection using Neural Networks in 5G Network Journal Article

In: International Journal for Research in Applied Science & Engineering Technology (IJRASET), vol. 9, iss. 12, 2021, ISSN: 2321-9653.

1678.

Ye, Andre

Successful Neural Network Architecture Design Book Chapter

In: Apress, Berkeley, CA, 2021.

1677.

Yan, C.; Chang, X.; Li, Z.; Guan, W.; Ge, Z.; Zhu, L.; Zheng, Q.

ZeroNAS: Differentiable Generative Adversarial Networks Search for Zero-Shot Learning Journal Article

In: IEEE Transactions on Pattern Analysis & Machine Intelligence, no. 01, pp. 1-1, 2021, ISSN: 1939-3539.

1676.

Wang, Shurun; Tang, Hao; Wang, Bin; Mo, Jia

A Novel Approach to Detecting Muscle Fatigue Based on sEMG by Using Neural Architecture Search Framework Proceedings Article

In: IEEE Trans Neural Netw Learn Syst, 2021.

1675.

Li, Siyi; Sun, Yanan; Yen, Gary G; Zhang, Mengjie

Automatic Design of Convolutional Neural Network Architectures Under Resource Constraints Proceedings Article

In: IEEE Trans Neural Netw Learn Syst ., 2021.

1674.

Nangi, Sharmila Reddy; Tyagi, Atharv; Mundra, Jay; Mukherjee, Sagnik; Snehal, Raj; Chhaya, Niyati; Garimella, Aparna

AUTOSUMM: Automatic Model Creation for Text Summarization Proceedings Article

In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10162–10172, Ässociation for Computational Linguistics, Önline and Punta Cana, Dominican Republic, 2021.

Abstract | Links | BibTeX

1673.

Liu, Zhengying

Automated Deep Learning : Principles and Practice PhD Thesis

2021.

1672.

Nguyen, N.; Chang, M. J.

CSNAS: Contrastive Self-supervised Learning Neural Architecture Search via Sequential Model-Based Optimization Journal Article

In: IEEE Transactions on Artificial Intelligence, vol. 1, no. 01, pp. 1-1, 2021.

4221 entries « ‹ 51 of 85 › »