Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2021
An Analysis of Super-Net Heuristics in Weight-Sharing NAS Journal Article
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.
Searching for Two-Stream Models in Multivariate Space for Video Recognition Technical Report
2021.
Bayesian Parametric and Architectural Domain Adaptation of LF-MMI Trained TDNNs for Elderly and Dysarthric Speech Recognition Proceedings Article
In: Proc. Interspeech 2021, pp. 4818–4822, 2021.
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Technical Report
2021.
In: ICT Express, 2021, ISSN: 2405-9595.
SMoA: Searching a Modality-Oriented Architecture for Infrared and Visible Image Fusion Journal Article
In: IEEE Signal Processing Letters, vol. 28, pp. 1818-1822, 2021.
Searching for Efficient Multi-Stage Vision Transformers Technical Report
2021.
Bayesian Hyperparameter Optimization of Deep Neural Network Algorithms Based on Ant Colony Optimization Proceedings Article
In: Lladós, Josep; Lopresti, Daniel; Uchida, Seiichi (Ed.): Document Analysis and Recognition -- ICDAR 2021, pp. 585–594, Springer International Publishing, Cham, 2021, ISBN: 978-3-030-86334-0.
Edge-featured Graph Neural Architecture Search Technical Report
2021.
Going Beyond Neural Architecture Search with Sampling-based Neural Ensemble Search Technical Report
2021.
NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization Technical Report
2021.
ISyNet: Convolutional Neural Networks design for AI accelerator Technical Report
2021.
A block-encoding method for evolving neural network architecture Journal Article
In: Int. J. Bio Inspired Comput., vol. 18, no. 1, pp. 27–37, 2021.
Automated Robustness with Adversarial Training as a Post-Processing Step Technical Report
2021.
ReconfigISP: Reconfigurable Camera Image Processing Pipeline Technical Report
2021.
Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry Technical Report
2021.
An Automatic Credit Scoring Strategy (ACSS) using memetic evolutionary algorithm and neural architecture search Journal Article
In: Applied Soft Computing, vol. 113, pp. 107871, 2021, ISSN: 1568-4946.
RepNAS: Searching for Efficient Re-parameterizing Blocks Technical Report
2021.
Rapid Model Architecture Adaption for Meta-Learning Technical Report
2021.
Domain Adaptation by Maximizing Population Correlation with Neural Architecture Search Technical Report
2021.
Accelerating DNN Architecture Search at Scale Using Selective Weight Transfer Proceedings Article
In: IEEE International Conference on Cluster Computing, CLUSTER 2021, Portland, OR, USA, September 7-10, 2021, pp. 82–93, IEEE, 2021.
Evolving Architectures with Gradient Misalignment toward Low Adversarial Transferability Technical Report
2021.
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking Technical Report
2021.
Self-Supervised Neural Architecture Search for Imbalanced Datasets Technical Report
2021.
Fast Deep Neural Architecture Search for Wearable Activity Recognition by Early Prediction of Converged Performance Proceedings Article
In: 2021 International Symposium on Wearable Computers, pp. 1–6, Association for Computing Machinery, Virtual, USA, 2021, ISBN: 9781450384629.
Neural Architecture Search Based on Evolutionary Algorithms with Fitness Approximation Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021.
AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks Technical Report
2021.
Search For Deep Graph Neural Networks Technical Report
2021.
SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification Proceedings Article
In: In Proceedings of IEEE Automatic Speech Recognition and Understanding (ASRU 2021), 2021, 2021.
G-CoS: GNN-Accelerator Co-Search Towards Both Better Accuracy and Efficiency Technical Report
2021.
Primer: Searching for Efficient Transformers for Language Modeling Technical Report
2021.
Enhancing Neural Architecture Search by Upgrading Weak Components Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021.
CellNet: An Improved Neural Architecture Search Method for Coal and Gangue Classification Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-9, 2021.
RNTS: Robust Neural Temporal Search for Time Series Classification Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021.
Cursor-based Adaptive Quantization for Deep Convolutional Neural Network Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021.
SSRNAS: Search Space Reduced One-shot NAS by a Recursive Attention-based Predictor with Cell Tensor-flow Diagram Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021.
Automated Graph Representation Learning for Node Classification Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-7, 2021.
Evolutionary NAS in Light of Model Stability for Accurate Continual Learning Proceedings Article
In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021.
Privacy-Preserving Multi-Granular Federated Neural Architecture Search A General Framework Journal Article
In: IEEE Transactions on Knowledge and Data Engineering, pp. 1-1, 2021.
Profiling Neural Blocks and Design Spaces for Mobile Neural Architecture Search Proceedings Article
In: Demartini, Gianluca; Zuccon, Guido; Culpepper, J. Shane; Huang, Zi; Tong, Hanghang (Ed.): CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021, pp. 4026–4035, ACM, 2021.
Forecasting of COVID-19 Cases, Using an Evolutionary Neural Architecture Search Approach Technical Report
2021.
A Feature Fusion Based Indicator for Training-Free Neural Architecture Search Journal Article
In: IEEE Access, vol. 9, pp. 133914-133923, 2021.
Delve into the Performance Degradation of Differentiable Architecture Search Proceedings Article
In: Demartini, Gianluca; Zuccon, Guido; Culpepper, J. Shane; Huang, Zi; Tong, Hanghang (Ed.): CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021, pp. 2547–2556, ACM, 2021.
L2NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning Proceedings Article
In: Demartini, Gianluca; Zuccon, Guido; Culpepper, J. Shane; Huang, Zi; Tong, Hanghang (Ed.): CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021, pp. 1284–1293, ACM, 2021.
Optimizing Neural Networks for Embedded Hardware Proceedings Article
In: SMACD / PRIME 2021; International Conference on SMACD and 16th Conference on PRIME, pp. 1-6, 2021.
Model Compression with NAS and Knowledge Distillation for Medical Image Segmentation Book Chapter
In: 2021 4th International Conference on Data Science and Information Technology, pp. 173–176, Association for Computing Machinery, New York, NY, USA, 2021, ISBN: 9781450390248.