AutoML.org

Freiburg-Hannover-Tübingen

Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

3298 entries « ‹ 6 of 66 › »

2023

3048.

(Ed.)

Designing a New Search Space for Multivariate Time-Series Neural Architecture Search Collection

2023.

3047.

García, Jesús Leopoldo Llano; Monroy, Raúl; Hernández, Víctor Adrián Sosa

An Experimental Protocol for Neural Architecture Search in Super-Resolution Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 4139-4146, 2023.

3046.

Soro, Bedionita; Song, Chong

Enhancing Differentiable Architecture Search: A Study on Small Number of Cell Blocks in the Search Stage, and Important Branches-Based Cells Selection Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1253-1261, 2023.

3045.

Rosales, Rafael; Munoz, Pablo; Paulitsch, Michael

Assessing the Impact of Diversity on the Resilience of Deep Learning Ensembles: A Comparative Study on Model Architecture, Output, Activation, and Attribution Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 4406-4416, 2023.

3044.

Bhardwaj, Kartikeya; Cheng, Hsin-Pai; Priyadarshi, Sweta; Li, Zhuojin

ZiCo-BC: A Bias Corrected Zero-Shot NAS for Vision Tasks Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1353-1357, 2023.

3043.

Cavagnero, Niccolò; Robbiano, Luca; Pistilli, Francesca; Caputo, Barbara; Averta, Giuseppe

Entropic Score Metric: Decoupling Topology and Size in Training-Free NAS Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1459-1468, 2023.

3042.

Sakuma, Yuiko; Ishii, Masato; Narihira, Takuya

DetOFA: Efficient Training of Once-for-All Networks for Object Detection Using Path Filter Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1333-1342, 2023.

3041.

Akinola, Solomon Oluwole; Qingguo, Wang; Olukanmi, Peter; Tshilidzi, Marwala

A Boosted Evolutionary Neural Architecture Search for Timeseries Forecasting with Application to South African COVID-19 Cases Journal Article

In: International Journal of Online and Biomedical Engineering (iJOE), vol. 19, no. 14, pp. pp. 107–130, 2023.

3040.

Zhang, Lei; Chen, Zhiqian; Lu, Chang-Tien; Zhao, Liang

Fast and Adaptive Dynamics-on-Graphs to Dynamics-of-Graphs Translation Journal Article

In: Sec. Data Mining and Management , 2023.

3039.

Hoang, Quang Minh

Practical Methods for Automated Algorithm Design in Machine Learning and Computational Biology PhD Thesis

2023.

3038.

An, S.; Channing, G.; Schuman, C.; Taufer, M.

VINARCH: A Visual Analytics Interactive Tool for Neural Network Archaeology Proceedings Article

In: 2023 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops), pp. 50-51, IEEE Computer Society, Los Alamitos, CA, USA, 2023.

Abstract | Links | BibTeX

3037.

Hu, Xuemei; Huang, Lan; Zeng, Jia; Wang, Kangping; Wang, Yan

EGFA-NAS: a neural architecture search method based on explosion gravitation field algorithm Journal Article

In: Complex & Intelligent Systems , 2023.

3036.

Pan, Yang; Jin, Mingwu; Zhang, Shun-Rong; Wing, Simon; Deng, Yue

Neural Network Models for Ionospheric Electron Density Prediction: A Neural Architecture Search Study Journal Article

In: ESS Open Archive, 2023.

3035.

O'Neill, Damien

Evolutionary Computation for the Optimisation of Skip-Connection Structures on Dense Convolutional Neural Networks PhD Thesis

2023.

3034.

(Ed.)

MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention Collection

2023.

3033.

(Ed.)

Automated Knowledge Distillation via Monte Carlo Tree Search Collection

2023.

3032.

Xu, Yang; Ma, Yongjie

Evolutionary neural architecture search combining multi-branch ConvNet and improved transformer Journal Article

In: 2023.

3031.

(Ed.)

An Evaluation of Zero-Cost Proxies - from Neural Architecture Performance Prediction to Model Robustness Collection

2023.

3030.

Rogers, Brendan; Noman, Nasimul; Chalup, Stephan; Moscato, Pablo

A comparative analysis of deep neural network architectures for sentence classification using genetic algorithm Journal Article

In: Evolutionary Intelligence, 2023.

3029.

Tang, Yi-Jun; Yan, Ke; Zhang, Xingyi; Tian, Ye; Liu, Bin

Protein intrinsically disordered region prediction by combining neural architecture search and multi-objective genetic algorithm Journal Article

In: BMC Biology , 2023.

3028.

Afif, Mouna; Ayachi, Riadh; Said, Yahia; Atri, Mohamed

An indoor scene recognition system based on deep learning evolutionary algorithms Journal Article

In: Soft Computing , 2023.

3027.

Zhang, Junfeng; Xie, Cheng; Yu, Beibei; Yang, Rui

Adaptive Hierarchical Knowledge Distillation from GNNs to MLPs Technical Report

2023.

3026.

Cheung, Ming

Learning from the Past: Fast NAS for Tasks and Datasets Journal Article

In: ACM Trans. Multimedia Comput. Commun. Appl., 2023, ISSN: 1551-6857, (Just Accepted).

Abstract | Links | BibTeX

3025.

Chitty-Venkata, Krishna Teja

Hardware-aware design, search, and optimization of deep neural networks Bachelor Thesis

2023.

3024.

Yang, Zhao; Sun, Qingshuang

Energy-Efficient Personalized Federated Search with Graph for Edge Computing Journal Article

In: ACM Trans. Embed. Comput. Syst., vol. 22, no. 5s, 2023, ISSN: 1539-9087.

Abstract | Links | BibTeX

3023.

Mousavi, Hamid; Loni, Mohammad; Alibeigi, Mina; Daneshtalab, Masoud

DASS: Differentiable Architecture Search for Sparse Neural Networks Journal Article

In: ACM Trans. Embed. Comput. Syst., vol. 22, no. 5s, 2023, ISSN: 1539-9087.

Abstract | Links | BibTeX

@article{10.1145/3609385,

title = {DASS: Differentiable Architecture Search for Sparse Neural Networks},

author = {Hamid Mousavi and Mohammad Loni and Mina Alibeigi and Masoud Daneshtalab},

url = {https://doi.org/10.1145/3609385},

doi = {10.1145/3609385},

issn = {1539-9087},

year  = {2023},

date = {2023-09-01},

urldate = {2023-09-01},

journal = {ACM Trans. Embed. Comput. Syst.},

volume = {22},

number = {5s},

publisher = {Association for Computing Machinery},

address = {New York, NY, USA},

abstract = {The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available computational power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current methods do not support sparse architectures in their search space and use a search objective that is made for dense networks and does not focus on sparsity.This paper proposes a new method to search for sparsity-friendly neural architectures. It is done by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that architectures found through DASS outperform those used in the state-of-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with a 3.87× faster inference time.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

3022.

Gao, Jianliang; Oloulade, Babatounde Moctard; Al-Sabri, Raeed; Chen, Jiamin; Lyu, Tengfei; zhenpeng Wu,

Graph neural architecture prediction Journal Article

In: Knowledge and Information Systems , 2023.

3021.

Huang, Lin; Deng, Weiming; Li, Chunchun; Yang, Tiejun

Object detection for blind inspection of industrial products based on neural architecture search Journal Article

In: 2023.

3020.

Barbudo, Rafael; Ventura, Sebastián; Romero, José Raúl

Eight years of AutoML: categorisation, review and trends Journal Article

In: Knowledge and Information Systems, 2023.

3019.

Lourens, Matt; Sinayskiy, Ilya; Park, Daniel K.; Blank, Carsten; Petruccione, Francesco

Hierarchical quantum circuit representations for neural architecture search Journal Article

In: npj Quantum Information , 2023.

3018.

Ang, Sui Paul; Phung, Son Lam; Duong, Soan T. M.; Bouzerdoum, Abdesselam

MSD-NAS: multi-scale dense neural architecture search for real-time pedestrian lane detection Bachelor Thesis

2023.

3017.

Lukasik, Jovita

Topology Learning for Prediction, Generation, and Robustness in Neural Architecture Search PhD Thesis

2023.

3016.

Li, Lintao; Jiang, Hongkai; Wang, Ruixin; Yang, Qiao

A reinforcement neural architecture search convolutional neural network for rolling bearing fault diagnosis Journal Article

In: Measurement Science and Technology, vol. 34, no. 11, pp. 115122, 2023.

Abstract | Links | BibTeX

3015.

Wu, Yixiao; Liu, Huan; Zhang, Dalin; Zhang, Yuzhe; Lou, Tianyu; Zheng, Qinghua

AutoEER: automatic EEG-based emotion recognition with neural architecture search Journal Article

In: Journal of Neural Engineering, vol. 20, no. 4, pp. 046029, 2023.

Abstract | Links | BibTeX

@article{Wu_2023,

title = {AutoEER: automatic EEG-based emotion recognition with neural architecture search},

author = {Yixiao Wu and Huan Liu and Dalin Zhang and Yuzhe Zhang and Tianyu Lou and Qinghua Zheng},

url = {https://dx.doi.org/10.1088/1741-2552/aced22},

doi = {10.1088/1741-2552/aced22},

year  = {2023},

date = {2023-08-01},

urldate = {2023-08-01},

journal = {Journal of Neural Engineering},

volume = {20},

number = {4},

pages = {046029},

publisher = {IOP Publishing},

abstract = {Objective. Emotion recognition based on electroencephalography (EEG) is garnering increasing attention among researchers due to its wide-ranging applications and the rise of portable devices. Deep learning-based models have demonstrated impressive progress in EEG-based emotion recognition, thanks to their exceptional feature extraction capabilities. However, the manual design of deep networks is time-consuming and labour-intensive. Moreover, the inherent variability of EEG signals necessitates extensive customization of models, exacerbating these challenges. Neural architecture search (NAS) methods can alleviate the need for excessive manual involvement by automatically discovering the optimal network structure for EEG-based emotion recognition. Approach. In this regard, we propose AutoEER (Automatic EEG-based Emotion Recognition), a framework that leverages tailored NAS to automatically discover the optimal network structure for EEG-based emotion recognition. We carefully design a customized search space specifically for EEG signals, incorporating operators that effectively capture both temporal and spatial properties of EEG. Additionally, we employ a novel parameterization strategy to derive the optimal network structure from the proposed search space. Main results. Extensive experimentation on emotion classification tasks using two benchmark datasets, DEAP and SEED, has demonstrated that AutoEER outperforms state-of-the-art manual deep and NAS models. Specifically, compared to the optimal model WangNAS on the accuracy (ACC) metric, AutoEER improves its average accuracy on all datasets by 0.93%. Similarly, compared to the optimal model LiNAS on the F1 Ssore (F1) metric, AutoEER improves its average F1 score on all datasets by 4.51%. Furthermore, the architectures generated by AutoEER exhibit superior transferability compared to alternative methods. Significance. AutoEER represents a novel approach to EEG analysis, utilizing a specialized search space to design models tailored to individual subjects. This approach significantly reduces the labour and time costs associated with manual model construction in EEG research, holding great promise for advancing the field and streamlining research practices.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Objective. Emotion recognition based on electroencephalography (EEG) is garnering increasing attention among researchers due to its wide-ranging applications and the rise of portable devices. Deep learning-based models have demonstrated impressive progress in EEG-based emotion recognition, thanks to their exceptional feature extraction capabilities. However, the manual design of deep networks is time-consuming and labour-intensive. Moreover, the inherent variability of EEG signals necessitates extensive customization of models, exacerbating these challenges. Neural architecture search (NAS) methods can alleviate the need for excessive manual involvement by automatically discovering the optimal network structure for EEG-based emotion recognition. Approach. In this regard, we propose AutoEER (Automatic EEG-based Emotion Recognition), a framework that leverages tailored NAS to automatically discover the optimal network structure for EEG-based emotion recognition. We carefully design a customized search space specifically for EEG signals, incorporating operators that effectively capture both temporal and spatial properties of EEG. Additionally, we employ a novel parameterization strategy to derive the optimal network structure from the proposed search space. Main results. Extensive experimentation on emotion classification tasks using two benchmark datasets, DEAP and SEED, has demonstrated that AutoEER outperforms state-of-the-art manual deep and NAS models. Specifically, compared to the optimal model WangNAS on the accuracy (ACC) metric, AutoEER improves its average accuracy on all datasets by 0.93%. Similarly, compared to the optimal model LiNAS on the F1 Ssore (F1) metric, AutoEER improves its average F1 score on all datasets by 4.51%. Furthermore, the architectures generated by AutoEER exhibit superior transferability compared to alternative methods. Significance. AutoEER represents a novel approach to EEG analysis, utilizing a specialized search space to design models tailored to individual subjects. This approach significantly reduces the labour and time costs associated with manual model construction in EEG research, holding great promise for advancing the field and streamlining research practices.

3014.

Rampavan, Medipelly; Ijjina, Earnest Paul

MNASreID: Grasshopper Optimization based Neural Architecture Search for Motorcycle re-IDentification Technical Report

2023.

3013.

Su, Xiu

Adapting Neural Architecture Search for Efficient Deep Learning Models PhD Thesis

2023.

3012.

Sridhar, Sharath Nittur; Kundu, Souvik; Sundaresan, Sairam; Szankin, Maciej; Sarah, Anthony

InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning Technical Report

2023.

3011.

ZiWen, Dou; YuQi, Li; Dong, Ye

FasterMDE: A real-time monocular depth estimation search method that balances accuracy and speed on the edge Journal Article

In: Applied Intelligence, 2023.

3010.

Hosseini, Ramtin; Zhang, Li; Garg, Bhanu; Xie, Pengtao

Fair and Accurate Decision Making through Group-Aware Learning Conference

ICML 2023, 2023.

3009.

Wang, Wei; Wang, Xianpeng; Song, Xiangman

Maximal sparse convex surrogate-assisted evolutionary convolutional neural architecture search for image segmentation Journal Article

In: Complex & Intelligent Systems , 2023.

3008.

Yang, Yi; Wei, Jiaxuan; Yu, Zhixuan; Zhang, Ruisheng

A trustworthy neural architecture search framework for pneumonia image classification utilizing blockchain technology Journal Article

In: The Journal of Supercomputing, 2023.

3007.

Liu, Ruihua; Nan, Haoyu; Zou, Yangyang; Xie, Ting

AS-3DFCN: Automatically Seeking 3DFCN-Based Brain Tumor Segmentation Conference

Cognitive Computation, 2023.

3006.

Thomson, Sarah L; Ochoa, Gabriela; Veerapen, Nadarajen; Michalak, Krzysztof

Channel Configuration for Neural Architecture: Insights from the Search Space Proceedings Article

In: The Genetic and Evolutionary Computation Conference (GECCO) 2023, 2023.

3005.

Kapoor, Rahul; Pillay, Nelishia

Iterative Structure-Based Genetic Programming for Neural Architecture Design Proceedings Article

In: GECCO ’23 Companion), 2023.

3004.

Zhang, Anqi; Zhao, Shengmei

Evolutionary-based searching method for quantum circuit architecture Journal Article

In: Quantum Information Processing , 2023.

3003.

(Ed.)

THE DYNAMIC PROBLEM OF AN ELASTICALLY SUPPORTED BEAM ANALYSIS USING NEURAL ARCHITECTURE SEARCH WITH AUTOMATIC DIFFERENTIATION Collection

2023.

3002.

Ren, Yaodanjun; Chen, Chen; Qi, Zhengwei

HaeNAS: Hardware-Aware Efficient Neural Architecture Search via Zero-Cost Proxy Miscellaneous

2023.

3001.

(Ed.)

SpeedLimit: Neural Architecture Search for Quantized Transformer Models Collection

2023.

3000.

(Ed.)

QuantumDARTS: Differentiable Quantum Architecture Search for Variational Quantum Algorithms Collection

2023.

2999.

(Ed.)

Data Driven Designing of Convolutional Neural Networks Architectures for Image Classification. Collection

2023.

3298 entries « ‹ 6 of 66 › »