AutoML.org

Freiburg-Hannover-Tübingen

Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

3251 entries « ‹ 36 of 66 › »

2021

1501.

Liu, Xiaobo; Zhang, Chaochao; Cai, Zhihua; Yang, Jianfeng; Zhou, Zhilang; Gong, Xin

Continuous Particle Swarm Optimization-Based Deep Learning Architecture Search for Hyperspectral Image Classification Journal Article

In: Remote Sensing, vol. 13, no. 6, 2021, ISSN: 2072-4292.

Abstract | Links | BibTeX

1500.

Ban, Hao; Xie, Pengtao

Interleaving Learning, with Application to Neural Architecture Search Technical Report

2021.

1499.

Choi, Kwanghee; Choe, Minyoung; Lee, Hyelee

Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning Technical Report

2021.

1498.

Gracheva, Ekaterina

Trainless Model Performance Estimation for Neural Architecture Search Technical Report

2021.

1497.

Wu, Xuan; Zhang, Xiuyi; Jia, Linhan; Chen, Liang; Liang, Yanchun; Zhou, You; Wu, Chunguo

Neural Architecture Search based on Cartesian Genetic Programming Coding Method Technical Report

2021.

1496.

Luo, Xiangzhong; Liu, Di; Huai, Shuo; Liu, Weichen

HSCoNAS: Hardware-Software Co-Design of Efficient DNNs via Neural Architecture Search Technical Report

2021.

1495.

Wang, Ruochen; Cheng, Minhao; Chen, Xiangning; Tang, Xiaocheng; Hsieh, Cho-Jui

Rethinking Architecture Selection in Differentiable NAS Proceedings Article

In: International Conference on Learning Representations, 2021.

1494.

-, Sian; -, Wei

Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator Technical Report

2021.

1493.

Lakhmiri, Dounia; é, S

Use of static surrogates in hyperparameter optimization Technical Report

2021.

1492.

Inkawhich, Matthew; Inkawhich, Nathan; Davis, Eric; Li, Hai; Chen, Yiran

The Untapped Potential of Off-the-Shelf Convolutional Neural Networks Technical Report

2021.

1491.

Chen, Zhiqiang; Xu, Ting-Bing; Liao, Weijian; Li, Zhengcheng; Li, Jinpeng; Liu, Cheng-Lin; He, Huiguang

SNAP: Shaping neural architectures progressively via information density criterion Journal Article

In: Pattern Recognition, vol. 116, pp. 107923, 2021, ISSN: 0031-3203.

Abstract | Links | BibTeX

@article{CHEN2021107923,

title = {SNAP: Shaping neural architectures progressively via information density criterion},

author = {Zhiqiang Chen and Ting-Bing Xu and Weijian Liao and Zhengcheng Li and Jinpeng Li and Cheng-Lin Liu and Huiguang He},

url = {https://www.sciencedirect.com/science/article/pii/S0031320321001102},

doi = {https://doi.org/10.1016/j.patcog.2021.107923},

issn = {0031-3203},

year  = {2021},

date = {2021-01-01},

journal = {Pattern Recognition},

volume = {116},

pages = {107923},

abstract = {Excellent neural network architecture is built on the specific target task and device. As the target task or device is different, the neural architecture we need will be different, too. Rather than redesigning or searching a brand new one, adjusting the existing architecture automatically is an alternative yet efficient way. To this end, we propose a method to Shape the existing Neural Architectures Progressively (SNAP) to adapt the target task and device better. Inspired by the streamline of water drop shaped by air resistance, we define an information density criterion (play the role of resistance) to drive the network architecture reducing the size of the part with the lowest information density. Iteratively, a more adaptive architecture will be obtained progressively in a greedy way. Theoretically, we prove that the greedy strategy is reasonable and can shape a better architecture. Because of the small adjustment of architecture each time, new architecture can inherit the parameters in old architecture to avoid retraining it from scratch. So the proposed method is very efficient in no need of high computation cost. Experimental results show that proposed method can effectively improve the given network by adjusting its architecture. And it can generate different architectures for different tasks and devices to adapt them well. Compared with search-based auto-generated neural architectures, our approach can achieve comparable or even better performance in no need of tremendous computation resources.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

1490.

Li, Chaojian; Yu, Zhongzhi; Fu, Yonggan; Zhang, Yongan; Zhao, Yang; You, Haoran; Yu, Qixuan; Wang, Yue; Lin, Yingyan

HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark Proceedings Article

In: ICLR 2021, 2021.

1489.

Pyeon, Myeongjang; Moon, Jihwan; Hahn, Taeyoung; Kim, Gunhee

SEDONA: Search for Decoupled Neural Networks toward Greedy Block-wise Learning Proceedings Article

In: International Conference on Learning Representations, 2021.

1488.

Zhou, Allan; Knowles, Tom; Finn, Chelsea

Meta-learning Symmetries by Reparameterization Proceedings Article

In: International Conference on Learning Representations, 2021.

1487.

Mehrotra, Abhinav; Ramos, Alberto Gil C P; Bhattacharya, Sourav; Ł,; Vipperla, Ravichander; Chau, Thomas; Abdelfattah, Mohamed S; Ishtiaq, Samin; Lane, Nicholas Donald

NAS-Bench-ASR: Reproducible Neural Architecture Search for Speech Recognition Proceedings Article

In: International Conference on Learning Representations, 2021.

1486.

Zhou, Daquan; Jin, Xiaojie; Lian, Xiaochen; Yang, Linjie; Xue, Yujing; Hou, Qibin; Feng, Jiashi

AutoSpace: Neural Architecture Search with Less Human Interference Technical Report

2021.

1485.

Li, Changlin; Tang, Tao; Wang, Guangrun; Peng, Jiefeng; Wang, Bing; Liang, Xiaodan; Chang, Xiaojun

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Technical Report

2021.

1484.

Abed, H; Gyires-Tóth, B

Efficient Neural Architecture Search for Long Short-Term Memory Networks Proceedings Article

In: 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 000287-000292, 2021.

1483.

Ai, Dige; Zhang, Hong

GNAS: A Generalized Neural Network Architecture Search Framework Technical Report

2021.

1482.

Zhang, Haichao; Hao, Kuangrong; Gao, Lei; -, Xue; Wei, Bing

Enhanced Gradient for Differentiable Architecture Search Technical Report

2021.

1481.

Su, Xiu; Huang, Tao; Li, Yanxi; You, Shan; Wang, Fei; Qian, Chen; Zhang, Changshui; Xu, Chang

Prioritized Architecture Sampling with Monto-Carlo Tree Search Technical Report

2021.

1480.

Wang, Hongbo; Pan, Xingyu; Fan, Lingyan; Zhao, Shuofeng

Steganalysis of convolutional neural network based on neural architecture search Journal Article

In: Multimedia Systems, 2021.

1479.

Lu, Zexin; Xia, Wenjun; Huang, Yongqiang; Shan, Hongming; Chen, Hu; Zhou, Jiliu; Zhang, Yi

MANAS: Multi-Scale and Multi-Level Neural Architecture Search for Low-Dose CT Denoising Journal Article

In: CoRR, vol. abs/2103.12995, 2021.

1478.

"Yang Zhao" "Hai Min" "Yan-Xiang Chen Wei Jia" "Wei Xia",

2D and 3D Palmprint and Palm Vein Recognition Based on Neural Architecture Search Journal Article

In: International Journal of Automation and Computing, no. IJAC-2021-01-01, pp. 1, 2021, ISSN: 1476-8186.

1477.

Lin, Yunfeng; Li, Guilin; Zhang, Xing; Zhang, Weinan; Chen, BO; Tang, Ruiming; Li, Zhenguo; Feng, Jiashi; Yu, YOng

ModularNAS: Towards Modularized and Reusable Neural Architecture Search Proceedings Article

In: Proceedings of Machine Learning and Systems 3 pre-proceedings (MLSys 2021), 2021.

1476.

Wang, Rui; Wei, Zhihua; Ji, Shouling; Hong, Zhen

EfficientTDNN: Efficient Architecture Search for Speaker Recognition in the Wild Technical Report

2021.

1475.

Ottelander, Tom Den; Dushatskiy, Arkadiy; Virgolin, Marco; Bosman, Peter A N

Local Search is a Remarkably Strong Baseline for Neural Architecture Search Proceedings Article

In: Ishibuchi, Hisao; Zhang, Qingfu; Cheng, Ran; Li, Ke; Li, Hui; Wang, Handing; Zhou, Aimin (Ed.): Evolutionary Multi-Criterion Optimization - 11th International Conference, EMO 2021, Shenzhen, China, March 28-31, 2021, Proceedings, pp. 465–479, Springer, 2021.

1474.

Ding, Mingyu; Huo, Yuqi; Lu, Haoyu; Yang, Linjie; Wang, Zhe; Lu, Zhiwu; Wang, Jingdong; Luo, Ping

Learning Versatile Neural Architectures by Propagating Network Codes Technical Report

2021.

1473.

Lankford, Séamus

Open-Source Neural Architecture Search with Ensemble and Pre-trained Networks Journal Article

In: International Journal of Modeling and Optimization, vol. 11, no. 2, pp. 33-41, 2021.

1472.

Li, Yanxi; Wen, Zean; Yunhe Wang,; Xu, Chang

One-shot Graph Neural Architecture Search with Dynamic Search Space Proceedings Article

In: AAAI 2021, 2021.

1471.

Cai, Shaofei; Li, Liang; Deng, Jincan; Zhang, Beichen; Zha, Zheng-Jun; Su, Li; Huang, Qingming

Rethinking Graph Neural Network Search from Message-passing Technical Report

2021.

1470.

Roberts, Nicholas; Khodak, Mikhail; Dao, Tri; Li, Liam; é, Christopher R; Talwalkar, Ameet

Rethinking Neural Operations for Diverse Tasks Technical Report

2021.

1469.

He, Yufan; Yang, Dong; Roth, Holger; Zhao, Can; Xu, Daguang

DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentation Technical Report

2021.

1468.

Schmidgall, Samuel

Self-Constructing Neural Networks Through Random Mutation Technical Report

2021.

1467.

Guo, Shaopeng; Wang, Yujie; Yuan, Kun; Li, Quanquan

Differentiable Network Adaption with Elastic Search Space Technical Report

2021.

1466.

Zappella, Giovanni; Salinas, David; Archambeau, Cédric

A resource-efficient method for repeated HPO and NAS problems Technical Report

2021.

1465.

Wang, J; Zhao, J; An, J; Wang, D; Qiu, M

A Linear NAS Service of ConvNets for Fast Deployment in the Edge of 5G Networks Journal Article

In: IEEE Network, vol. 35, no. 2, pp. 46-52, 2021.

1464.

Lee, Hayeon; Hyung, Eunyoung; Hwang, Sung Ju

Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets Proceedings Article

In: International Conference on Learning Representations, 2021.

1463.

Ding, Z; Chen, Y; Li, N; Zhao, D; Sun, Z; Chen, C L P

BNAS: Efficient Neural Architecture Search Using Broad Scalable Architecture Journal Article

In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2021.

1462.

-, Tien; -, Yi; Sze, Vivienne

NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization Technical Report

2021.

1461.

Hu, Shoukang; Xie, Xurong; Liu, Shansong; Cui, Mingyu; Geng, Mengzhe; Liu, Xunying; Meng, Helen

Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks Proceedings Article

In: ICASSP2021, 2021.

1460.

Wang, X; Wang, X; Jin, L; Lv, R; Dai, B; He, M; Lv, T

Evolutionary Algorithm-Based and Network Architecture Search-Enabled Multiobjective Traffic Classification Journal Article

In: IEEE Access, vol. 9, pp. 52310-52325, 2021.

1459.

Wang, Guangrun; Lin, Liang; Chen, Rongcong; Wang, Guangcong; Zhang, Jiqi

Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition Journal Article

In: IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2021.

1458.

Prabakaran, B S; Akhtar, A; Rehman, S; Hasan, O; Shafique, M

BioNetExplorer: Architecture-Space Exploration of Bio-Signal Processing Deep Neural Networks for Wearables Journal Article

In: IEEE Internet of Things Journal, pp. 1-1, 2021.

1457.

Tan, Mingxing; Le, Quoc V

EfficientNetV2: Smaller Models and Faster Training Technical Report

2021.

1456.

Liang, Y; Lu, L; Jin, Y; Xie, J; Huang, R; Zhang, J; Lin, W

An Efficient Hardware Design for Accelerating Sparse CNNs with NAS-based Models Journal Article

In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2021.

1455.

ElSaid, AbdElRahman A.; Karns, Joshua; Lyu, Zimeng; II, Alexander G. Ororbia; Desell, Travis

Continuous Ant-Based Neural Topology Search Proceedings Article

In: Castillo, Pedro A.; Laredo, Juan Luis Jim'enez (Ed.): Applications of Evolutionary Computation: 24th International Conference, EvoApplications 2021, Held as Part of EvoStar 2021, Virtual Event, April 7--9, 2021, Proceedings, pp. 291, Springer Nature 2021.

1454.

White, Colin; Zela, Arber; Ru, Binxin; Liu, Yang; Hutter, Frank

How Powerful are Performance Predictors in Neural Architecture Search? Technical Report

2021.

1453.

Ren, Pengzhen; Xiao, Gang; Chang, Xiaojun; Xiao, Yun; Li, Zhihui; Chen, Xiaojiang

NAS-TC: Neural Architecture Search on Temporal Convolutions for Complex Action Recognition Technical Report

2021.

1452.

Dong, Heyou; Chen, Dan; Zhang, Lei; Ke, Hengjin; Li, Xiaoli

Subject Sensitive EEG Discrimination with Fast Reconstructable CNN Driven by Reinforcement Learning: A Case Study of ASD Evaluation Journal Article

In: Neurocomputing, 2021, ISSN: 0925-2312.

Abstract | Links | BibTeX

@article{DONG2021,

title = {Subject Sensitive EEG Discrimination with Fast Reconstructable CNN Driven by Reinforcement Learning: A Case Study of ASD Evaluation},

author = {Heyou Dong and Dan Chen and Lei Zhang and Hengjin Ke and Xiaoli Li},

url = {https://www.sciencedirect.com/science/article/pii/S0925231221005270},

doi = {https://doi.org/10.1016/j.neucom.2021.04.009},

issn = {0925-2312},

year  = {2021},

date = {2021-01-01},

journal = {Neurocomputing},

abstract = {Recent Electroencephalogram (EEG) analysis in connection with brain disorders has been tremendously benefiting from the (Deep) Neural Network technology in neuroscience research and neuro-engineering practices. However, the performance of existing hand-crafted models, such as the stability, has largely been refrained. This is the case especially in the paradigms that sensitive to the individuality of subjects and the non-stationarity of cognitive dynamics, such as Autism Spectrum Disorder (ASD) evaluation. Aiming at this problem, this study develops a Q-Learning method to enable fast reconstruction of Convolutional Neural Network (CNN) thus to support EEG discrimination adapting to the individuality of subjects under examination. The proposed method first generates a CNN model with the structure and hyper-parameters determined (i.e., Neural Architecture Search) by the customized Q-Learning algorithm, where the CNN model is treated as a discrete system to be optimized. With the sharp shift of subjects, the Q-Learning algorithm reconstructs the CNN model to reach optimization reusing the tacit knowledge learned from the previous trials. A case study has been performed to check the proposed method versus state-of-the-art counterparts based on resting-state EEG collected from 175 ASD-suspicious children with a diverse geological distribution. The observations in the case study indicate that: 1) the method outperforms the counterparts with an individual/sample accuracy of 92.63%/83.23% achieved; 2) the method can quickly reconstruct the CNN model with the group of subjects shifting from one region to another to maintain an encouraging performance while the counterparts without reconstruction may drop by about 12%.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

3251 entries « ‹ 36 of 66 › »