Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
2786.
Hosseini, Ramtin; Zhang, Li; Garg, Bhanu; Xie, Pengtao
Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-00486,
title = {Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy},
author = {Ramtin Hosseini and Li Zhang and Bhanu Garg and Pengtao Xie},
url = {https://doi.org/10.48550/arXiv.2304.00486},
doi = {10.48550/arXiv.2304.00486},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.00486},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2785.
Wu, Meng-Ting; Lin, Hung-I; Tsai, Chun-Wei
A Training-Free Neural Architecture Search Algorithm based on Search Economics Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2023.
@article{10092788,
title = {A Training-Free Neural Architecture Search Algorithm based on Search Economics},
author = {Meng-Ting Wu and Hung-I Lin and Chun-Wei Tsai},
doi = {10.1109/TEVC.2023.3264533},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2784.
Karras, Aristeidis; Karras, Christos; Schizas, Nikolaos; Avlonitis, Markos; Sioutas, Spyros
AutoML with Bayesian Optimizations for Big Data Management Journal Article
In: Information, vol. 14, no. 4, 2023, ISSN: 2078-2489.
@article{info14040223,
title = {AutoML with Bayesian Optimizations for Big Data Management},
author = {Aristeidis Karras and Christos Karras and Nikolaos Schizas and Markos Avlonitis and Spyros Sioutas},
url = {https://www.mdpi.com/2078-2489/14/4/223},
doi = {10.3390/info14040223},
issn = {2078-2489},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Information},
volume = {14},
number = {4},
abstract = {The field of automated machine learning (AutoML) has gained significant attention in recent years due to its ability to automate the process of building and optimizing machine learning models. However, the increasing amount of big data being generated has presented new challenges for AutoML systems in terms of big data management. In this paper, we introduce Fabolas and learning curve extrapolation as two methods for accelerating hyperparameter optimization. Four methods for quickening training were presented including Bag of Little Bootstraps, k-means clustering for Support Vector Machines, subsample size selection for gradient descent, and subsampling for logistic regression. Additionally, we also discuss the use of Markov Chain Monte Carlo (MCMC) methods and other stochastic optimization techniques to improve the efficiency of AutoML systems in managing big data. These methods enhance various facets of the training process, making it feasible to combine them in diverse ways to gain further speedups. We review several combinations that have potential and provide a comprehensive understanding of the current state of AutoML and its potential for managing big data in various industries. Furthermore, we also mention the importance of parallel computing and distributed systems to improve the scalability of the AutoML systems while working with big data.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The field of automated machine learning (AutoML) has gained significant attention in recent years due to its ability to automate the process of building and optimizing machine learning models. However, the increasing amount of big data being generated has presented new challenges for AutoML systems in terms of big data management. In this paper, we introduce Fabolas and learning curve extrapolation as two methods for accelerating hyperparameter optimization. Four methods for quickening training were presented including Bag of Little Bootstraps, k-means clustering for Support Vector Machines, subsample size selection for gradient descent, and subsampling for logistic regression. Additionally, we also discuss the use of Markov Chain Monte Carlo (MCMC) methods and other stochastic optimization techniques to improve the efficiency of AutoML systems in managing big data. These methods enhance various facets of the training process, making it feasible to combine them in diverse ways to gain further speedups. We review several combinations that have potential and provide a comprehensive understanding of the current state of AutoML and its potential for managing big data in various industries. Furthermore, we also mention the importance of parallel computing and distributed systems to improve the scalability of the AutoML systems while working with big data.
2783.
Mishra, Vidyanand; Kane, Lalit
An evolutionary framework for designing adaptive convolutional neural network Journal Article
In: Expert Systems with Applications, vol. 224, pp. 120032, 2023, ISSN: 0957-4174.
@article{MISHRA2023120032,
title = {An evolutionary framework for designing adaptive convolutional neural network},
author = {Vidyanand Mishra and Lalit Kane},
url = {https://www.sciencedirect.com/science/article/pii/S0957417423005341},
doi = {https://doi.org/10.1016/j.eswa.2023.120032},
issn = {0957-4174},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Expert Systems with Applications},
volume = {224},
pages = {120032},
abstract = {The Convolutional Neural Network (CNN) is a complex architecture that performs magnificently in image classification and segmentation problems. Still, selecting an effective architecture is typically hindered by several parameters. Empirically, evolutionary algorithms (EA) have been found adequate in parameter selection and automated neural network search. However, the huge computational requirements imposed by evolutionary search make its applicability unexplored. Consequently, the idea of a CNN architecture selection based on EA is challenging as comparing complex candidate architectures towards their fitness would involve massive computations. In this work, we propose a novel framework using an adapted Genetic Algorithm (GA) that automatically evolves an effective CNN architecture. We rectify the GA by devising an effective encoding scheme, an approach to initialize the input population, and a diversified offspring generation method. We also suggest an optimized fitness function that makes the convergence faster, avoiding the local optima. The method is validated with the benchmark MNIST, Fashion_MNIST, and CIFAR-10 datasets. The results are comparable to the best manual and automatic state-of-the-art architectures regarding accuracy, convergence rate, and consumed computation resources.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The Convolutional Neural Network (CNN) is a complex architecture that performs magnificently in image classification and segmentation problems. Still, selecting an effective architecture is typically hindered by several parameters. Empirically, evolutionary algorithms (EA) have been found adequate in parameter selection and automated neural network search. However, the huge computational requirements imposed by evolutionary search make its applicability unexplored. Consequently, the idea of a CNN architecture selection based on EA is challenging as comparing complex candidate architectures towards their fitness would involve massive computations. In this work, we propose a novel framework using an adapted Genetic Algorithm (GA) that automatically evolves an effective CNN architecture. We rectify the GA by devising an effective encoding scheme, an approach to initialize the input population, and a diversified offspring generation method. We also suggest an optimized fitness function that makes the convergence faster, avoiding the local optima. The method is validated with the benchmark MNIST, Fashion_MNIST, and CIFAR-10 datasets. The results are comparable to the best manual and automatic state-of-the-art architectures regarding accuracy, convergence rate, and consumed computation resources.
2782.
Lee, Bokyeung; Kim, Donghyeon; Kim, Gwantae; Ko, Hanseok
Channel Shuffle Neural Architecture Search for Key Word Spotting Journal Article
In: IEEE Signal Processing Letters, vol. 30, pp. 443-447, 2023.
@article{10097672,
title = {Channel Shuffle Neural Architecture Search for Key Word Spotting},
author = {Bokyeung Lee and Donghyeon Kim and Gwantae Kim and Hanseok Ko},
doi = {10.1109/LSP.2023.3265573},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Signal Processing Letters},
volume = {30},
pages = {443-447},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2781.
Zhu, Xunyu; Li, Jian; Liu, Yong; Wang, Weiping
Robust Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-02845,
title = {Robust Neural Architecture Search},
author = {Xunyu Zhu and Jian Li and Yong Liu and Weiping Wang},
url = {https://doi.org/10.48550/arXiv.2304.02845},
doi = {10.48550/arXiv.2304.02845},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.02845},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2780.
Xie, Beini; Chang, Heng; Zhang, Ziwei; Wang, Xin; Wang, Daixin; Zhang, Zhiqiang; Ying, Rex; Zhu, Wenwu
Adversarially Robust Neural Architecture Search for Graph Neural Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-04168,
title = {Adversarially Robust Neural Architecture Search for Graph Neural Networks},
author = {Beini Xie and Heng Chang and Ziwei Zhang and Xin Wang and Daixin Wang and Zhiqiang Zhang and Rex Ying and Wenwu Zhu},
url = {https://doi.org/10.48550/arXiv.2304.04168},
doi = {10.48550/arXiv.2304.04168},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.04168},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2779.
Bingham, Garrett
Optimizing Neural Networks through Activation Function Discovery and Automatic Weight Initialization PhD Thesis
2023.
@phdthesis{DBLP:journals/corr/abs-2304-03374,
title = {Optimizing Neural Networks through Activation Function Discovery and Automatic Weight Initialization},
author = {Garrett Bingham},
url = {https://doi.org/10.48550/arXiv.2304.03374},
doi = {10.48550/arXiv.2304.03374},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.03374},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2778.
Heuillet, Alexandre; Nasser, Ahmad; Arioui, Hichem; Tabia, Hedi
Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-05405,
title = {Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search},
author = {Alexandre Heuillet and Ahmad Nasser and Hichem Arioui and Hedi Tabia},
url = {https://doi.org/10.48550/arXiv.2304.05405},
doi = {10.48550/arXiv.2304.05405},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.05405},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2777.
Jiang, Wenbin; Chen, Yuhao; Wen, Suyang; Zheng, Long; Jin, Hai
PDAS: Improving network pruning based on Progressive Differentiable Architecture Search for DNNs Journal Article
In: Future Generation Computer Systems, vol. 146, pp. 98-113, 2023, ISSN: 0167-739X.
@article{JIANG202398,
title = {PDAS: Improving network pruning based on Progressive Differentiable Architecture Search for DNNs},
author = {Wenbin Jiang and Yuhao Chen and Suyang Wen and Long Zheng and Hai Jin},
url = {https://www.sciencedirect.com/science/article/pii/S0167739X23001462},
doi = {https://doi.org/10.1016/j.future.2023.04.011},
issn = {0167-739X},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Future Generation Computer Systems},
volume = {146},
pages = {98-113},
abstract = {Network pruning is an efficient approach to adapting large-scale deep neural networks (DNNs) to resource-constrained systems; the networks are pruned using the predefined pruning criteria or a flexible network structure is explored with the help of neural architecture search, (NAS). However, the former crucially relies on the human expert knowledge, while the latter usually requires one to make many simplifications to ensure the efficiency of the search, resulting in limited performance. This paper presents a new pruning approach called Progressive Differentiable Architecture Search (PDAS) that realizes a better balance between computation efficiency and model performance. First, a joint search-update scheme for search optimization is presented; it constantly refines the candidate number of channels in each layer by performing differentiable searching and evolutionary updating alternately. The latter can provide new high-probability candidates continuously to avoid local minimum point. Second, a two-stage constrained progressive search strategy is presented for some complex nonlinear networks (such as ResNet) that are more difficult to prune for existing approaches; it effectively avoids the over-fitting problem caused by the excessive search space and largely reduces the consumption of 1x1 convolution in the skip connections of the residual blocks with little accuracy loss. Extensive experiments on some representative datasets (such as CIFAR-10, CIFAR-100, and ImageNet) approve the superior performance of PDAS compared to most existing network pruning algorithms available. Notably, compared to the state-of-the-art LFPC, PDAS can even prune about 8% more FLOPs on ResNet-110 (on CIFAR-10) and ResNet-50 (on ImageNet), respectively, while coming with almost identical and ignorable accuracy losses.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Network pruning is an efficient approach to adapting large-scale deep neural networks (DNNs) to resource-constrained systems; the networks are pruned using the predefined pruning criteria or a flexible network structure is explored with the help of neural architecture search, (NAS). However, the former crucially relies on the human expert knowledge, while the latter usually requires one to make many simplifications to ensure the efficiency of the search, resulting in limited performance. This paper presents a new pruning approach called Progressive Differentiable Architecture Search (PDAS) that realizes a better balance between computation efficiency and model performance. First, a joint search-update scheme for search optimization is presented; it constantly refines the candidate number of channels in each layer by performing differentiable searching and evolutionary updating alternately. The latter can provide new high-probability candidates continuously to avoid local minimum point. Second, a two-stage constrained progressive search strategy is presented for some complex nonlinear networks (such as ResNet) that are more difficult to prune for existing approaches; it effectively avoids the over-fitting problem caused by the excessive search space and largely reduces the consumption of 1x1 convolution in the skip connections of the residual blocks with little accuracy loss. Extensive experiments on some representative datasets (such as CIFAR-10, CIFAR-100, and ImageNet) approve the superior performance of PDAS compared to most existing network pruning algorithms available. Notably, compared to the state-of-the-art LFPC, PDAS can even prune about 8% more FLOPs on ResNet-110 (on CIFAR-10) and ResNet-50 (on ImageNet), respectively, while coming with almost identical and ignorable accuracy losses.
2776.
Mo, Hyunho; Iacca, Giovanni
Evolutionary neural architecture search on transformers for RUL prediction Journal Article
In: Materials and Manufacturing Processes, vol. 0, no. 0, pp. 1-18, 2023.
@article{doi:10.1080/10426914.2023.2199499,
title = {Evolutionary neural architecture search on transformers for RUL prediction},
author = {Hyunho Mo and Giovanni Iacca},
url = {https://doi.org/10.1080/10426914.2023.2199499},
doi = {10.1080/10426914.2023.2199499},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Materials and Manufacturing Processes},
volume = {0},
number = {0},
pages = {1-18},
publisher = {Taylor & Francis},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2775.
Fang, Wei; Zhu, Zhenhao; Zhu, Shuwei; Sun, Jun; Wu, Xiaojun; Lu, Zhichao
LoNAS: Low-Cost Neural Architecture Search Using a Three-Stage Evolutionary Algorithm [Research Frontier] Journal Article
In: IEEE Computational Intelligence Magazine, vol. 18, no. 2, pp. 78-93, 2023.
@article{10102389,
title = {LoNAS: Low-Cost Neural Architecture Search Using a Three-Stage Evolutionary Algorithm [Research Frontier]},
author = {Wei Fang and Zhenhao Zhu and Shuwei Zhu and Jun Sun and Xiaojun Wu and Zhichao Lu},
url = {https://ieeexplore.ieee.org/abstract/document/10102389},
doi = {10.1109/MCI.2023.3245799},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Computational Intelligence Magazine},
volume = {18},
number = {2},
pages = {78-93},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2774.
Du, Qianjin; Kuang, Xiaohui; Li, Xiang; Zhao, Gang
Fine-Grained Software Vulnerability Detection via Neural Architecture Search Proceedings Article
In: Wang, Xin; Sapino, Maria Luisa; Han, Wook-Shin; Abbadi, Amr El; Dobbie, Gill; Feng, Zhiyong; Shao, Yingxiao; Yin, Hongzhi (Ed.): Database Systems for Advanced Applications, pp. 224–238, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-30678-5.
@inproceedings{10.1007/978-3-031-30678-5_17,
title = {Fine-Grained Software Vulnerability Detection via Neural Architecture Search},
author = {Qianjin Du and Xiaohui Kuang and Xiang Li and Gang Zhao},
editor = {Xin Wang and Maria Luisa Sapino and Wook-Shin Han and Amr El Abbadi and Gill Dobbie and Zhiyong Feng and Yingxiao Shao and Hongzhi Yin},
url = {https://link.springer.com/chapter/10.1007/978-3-031-30678-5_17},
isbn = {978-3-031-30678-5},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Database Systems for Advanced Applications},
pages = {224--238},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Vulnerability detection methods based on the deep learning have achieved remarkable performance improvements compared to traditional methods. Current deep learning-based detectors mostly use a single RNN or its variants (i.e., LSTM or GRU) to detect vulnerabilities. However, vulnerability detection is a multi-domain problem. Different types of vulnerabilities have different characteristics. Using a single neural network cannot perform well in all types of vulnerability detection tasks. Manually designing a matching neural network for each type of vulnerability detection task not only requires a lot of trials and computational resources but also highly relies on the knowledge and design experience of experts. To address this issue, in this paper, we propose a novel fine-grained vulnerability detection framework named A-DARTS, which is capable of searching for well-performed neural network architectures for different vulnerability detection tasks by introducing neural network architecture search (NAS) techniques. Specifically, we design a more efficient search space to ensure superior neural network architectures can be found by the search algorithm. Besides, we propose an adaptive differentiable search algorithm to search for superior neural network architectures. Experimental results show that searched models consistently outperform all baseline models and achieve significant performance improvements.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Vulnerability detection methods based on the deep learning have achieved remarkable performance improvements compared to traditional methods. Current deep learning-based detectors mostly use a single RNN or its variants (i.e., LSTM or GRU) to detect vulnerabilities. However, vulnerability detection is a multi-domain problem. Different types of vulnerabilities have different characteristics. Using a single neural network cannot perform well in all types of vulnerability detection tasks. Manually designing a matching neural network for each type of vulnerability detection task not only requires a lot of trials and computational resources but also highly relies on the knowledge and design experience of experts. To address this issue, in this paper, we propose a novel fine-grained vulnerability detection framework named A-DARTS, which is capable of searching for well-performed neural network architectures for different vulnerability detection tasks by introducing neural network architecture search (NAS) techniques. Specifically, we design a more efficient search space to ensure superior neural network architectures can be found by the search algorithm. Besides, we propose an adaptive differentiable search algorithm to search for superior neural network architectures. Experimental results show that searched models consistently outperform all baseline models and achieve significant performance improvements.
2773.
Liu, Xukun
The Utilities of Evolutionary Multiobjective Optimization for Neural Architecture Search -- An Empirical Perspective Proceedings Article
In: Pan, Linqiang; Zhao, Dongming; Li, Lianghao; Lin, Jianqing (Ed.): Bio-Inspired Computing: Theories and Applications, pp. 179–195, Springer Nature Singapore, Singapore, 2023, ISBN: 978-981-99-1549-1.
@inproceedings{10.1007/978-981-99-1549-1_15,
title = {The Utilities of Evolutionary Multiobjective Optimization for Neural Architecture Search -- An Empirical Perspective},
author = {Xukun Liu},
editor = {Linqiang Pan and Dongming Zhao and Lianghao Li and Jianqing Lin},
isbn = {978-981-99-1549-1},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Bio-Inspired Computing: Theories and Applications},
pages = {179--195},
publisher = {Springer Nature Singapore},
address = {Singapore},
abstract = {Evolutionary algorithms have been widely used in neural architecture search (NAS) in recent years due to their flexible frameworks and promising performance. However, we noticed a lack of attention to algorithm selection, and single-objective algorithms were preferred despite the multiobjective nature of NAS, among prior arts. To explore the reasons behind this preference, we tested mainstream evolutionary algorithms on several standard NAS benchmarks, comparing single and multi-objective algorithms. Additionally, we validated whether the latest evolutionary multi-objective optimization (EMO) algorithms lead to improvement in NAS problems compared to classical EMO algorithms. Our experimental results provide empirical answers to these questions and guidance for the future development of evolutionary NAS algorithms.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Evolutionary algorithms have been widely used in neural architecture search (NAS) in recent years due to their flexible frameworks and promising performance. However, we noticed a lack of attention to algorithm selection, and single-objective algorithms were preferred despite the multiobjective nature of NAS, among prior arts. To explore the reasons behind this preference, we tested mainstream evolutionary algorithms on several standard NAS benchmarks, comparing single and multi-objective algorithms. Additionally, we validated whether the latest evolutionary multi-objective optimization (EMO) algorithms lead to improvement in NAS problems compared to classical EMO algorithms. Our experimental results provide empirical answers to these questions and guidance for the future development of evolutionary NAS algorithms.
2772.
Zhao, Chenggang; Zhang, Genghan; Gao, Mingyu
Canvas: End-to-End Kernel Architecture Search in Neural Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-07741,
title = {Canvas: End-to-End Kernel Architecture Search in Neural Networks},
author = {Chenggang Zhao and Genghan Zhang and Mingyu Gao},
url = {https://doi.org/10.48550/arXiv.2304.07741},
doi = {10.48550/arXiv.2304.07741},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.07741},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2771.
Kerssies, Tommie
Neural Architecture Search for Visual Anomaly Segmentation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-08975,
title = {Neural Architecture Search for Visual Anomaly Segmentation},
author = {Tommie Kerssies},
url = {https://doi.org/10.48550/arXiv.2304.08975},
doi = {10.48550/arXiv.2304.08975},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.08975},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2770.
Liu, Yang; Yan, Shen; Zhang, Yuge; Ren, Kan; Zhang, Quanlu; Ren, Zebin; Cai, Deng; Zhang, Mi
AutoTaskFormer: Searching Vision Transformers for Multi-task Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-08756,
title = {AutoTaskFormer: Searching Vision Transformers for Multi-task Learning},
author = {Yang Liu and Shen Yan and Yuge Zhang and Kan Ren and Quanlu Zhang and Zebin Ren and Deng Cai and Mi Zhang},
url = {https://doi.org/10.48550/arXiv.2304.08756},
doi = {10.48550/arXiv.2304.08756},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.08756},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2769.
Zheng, Mingkai; Su, Xiu; You, Shan; Wang, Fei; Qian, Chen; Xu, Chang; Albanie, Samuel
Can GPT-4 Perform Neural Architecture Search? Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-10970,
title = {Can GPT-4 Perform Neural Architecture Search?},
author = {Mingkai Zheng and Xiu Su and Shan You and Fei Wang and Chen Qian and Chang Xu and Samuel Albanie},
url = {https://doi.org/10.48550/arXiv.2304.10970},
doi = {10.48550/arXiv.2304.10970},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.10970},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2768.
Fan, Yicheng; Alon, Dana; Shen, Jingyue; Peng, Daiyi; Kumar, Keshav; Long, Yun; Wang, Xin; Iliopoulos, Fotis; Juan, Da-Cheng; Vee, Erik
LayerNAS: Neural Architecture Search in Polynomial Complexity Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11517,
title = {LayerNAS: Neural Architecture Search in Polynomial Complexity},
author = {Yicheng Fan and Dana Alon and Jingyue Shen and Daiyi Peng and Keshav Kumar and Yun Long and Xin Wang and Fotis Iliopoulos and Da-Cheng Juan and Erik Vee},
url = {https://doi.org/10.48550/arXiv.2304.11517},
doi = {10.48550/arXiv.2304.11517},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11517},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2767.
Wang, Di; Du, Bo; Zhang, Liangpei; Tao, Dacheng
HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11701,
title = {HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search},
author = {Di Wang and Bo Du and Liangpei Zhang and Dacheng Tao},
url = {https://doi.org/10.48550/arXiv.2304.11701},
doi = {10.48550/arXiv.2304.11701},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11701},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2766.
Li, Chao; Xu, Hao; He, Kun
Meta-multigraph Search: Rethinking Meta-structure on Heterogeneous Information Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11574,
title = {Meta-multigraph Search: Rethinking Meta-structure on Heterogeneous Information Networks},
author = {Chao Li and Hao Xu and Kun He},
url = {https://doi.org/10.48550/arXiv.2304.11574},
doi = {10.48550/arXiv.2304.11574},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11574},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2765.
Fu, Yonggan; Li, Yuecheng; Li, Chenghui; Saragih, Jason M.; Zhang, Peizhao; Dai, Xiaoliang; Lin, Yingyan
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11835,
title = {Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence},
author = {Yonggan Fu and Yuecheng Li and Chenghui Li and Jason M. Saragih and Peizhao Zhang and Xiaoliang Dai and Yingyan Lin},
url = {https://doi.org/10.48550/arXiv.2304.11835},
doi = {10.48550/arXiv.2304.11835},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11835},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2764.
Li, YuFei; Wu, Jia; Deng, TianJin
Meta-GNAS: Meta-reinforcement learning for graph neural architecture search Journal Article
In: Engineering Applications of Artificial Intelligence, vol. 123, pp. 106300, 2023, ISSN: 0952-1976.
@article{LI2023106300,
title = {Meta-GNAS: Meta-reinforcement learning for graph neural architecture search},
author = {YuFei Li and Jia Wu and TianJin Deng},
url = {https://www.sciencedirect.com/science/article/pii/S0952197623004840},
doi = {https://doi.org/10.1016/j.engappai.2023.106300},
issn = {0952-1976},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Engineering Applications of Artificial Intelligence},
volume = {123},
pages = {106300},
abstract = {Graph Neural architecture search (GNAS) has shown great success in designing many prominent models on non-Euclidean data. However, the existing GNAS methods need to search from scratch on new tasks, which is time-consuming and inefficient in real application scenarios. In this paper, we propose a meta-reinforcement learning method for Graph Neural Architecture Search (Meta-GNAS) to improve the learning efficiency on new tasks by leveraging the knowledge learned from previous tasks. As far as we know, it is the first work that applies meta-learning to GNAS tasks. Moreover, to further improve the efficiency in tackling a new task, we use a predictive model to evaluate the accuracy of sampled graph neural architecture, instead of training it from scratch. The experiment results demonstrate that the architecture designed by Meta-GNAS outperforms the state-of-art manually designed architectures, and the search speed is faster than other search methods, with an average search time of fewer than 210 GPU seconds on 6 datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Graph Neural architecture search (GNAS) has shown great success in designing many prominent models on non-Euclidean data. However, the existing GNAS methods need to search from scratch on new tasks, which is time-consuming and inefficient in real application scenarios. In this paper, we propose a meta-reinforcement learning method for Graph Neural Architecture Search (Meta-GNAS) to improve the learning efficiency on new tasks by leveraging the knowledge learned from previous tasks. As far as we know, it is the first work that applies meta-learning to GNAS tasks. Moreover, to further improve the efficiency in tackling a new task, we use a predictive model to evaluate the accuracy of sampled graph neural architecture, instead of training it from scratch. The experiment results demonstrate that the architecture designed by Meta-GNAS outperforms the state-of-art manually designed architectures, and the search speed is faster than other search methods, with an average search time of fewer than 210 GPU seconds on 6 datasets.
2763.
Xiao, Anqi; Shen, Biluo; Tian, Jie; Hu, Zhenhua
PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks Journal Article
In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-13, 2023.
@article{10109323,
title = {PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks},
author = {Anqi Xiao and Biluo Shen and Jie Tian and Zhenhua Hu},
doi = {10.1109/TNNLS.2023.3264551},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2762.
Liu, Yong; Zhou, Xin; Zhong, Wei
Multi-Modality Image Fusion and Object Detection Based on Semantic Information Journal Article
In: Entropy, vol. 25, no. 5, 2023, ISSN: 1099-4300.
@article{e25050718,
title = {Multi-Modality Image Fusion and Object Detection Based on Semantic Information},
author = {Yong Liu and Xin Zhou and Wei Zhong},
url = {https://www.mdpi.com/1099-4300/25/5/718},
doi = {10.3390/e25050718},
issn = {1099-4300},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Entropy},
volume = {25},
number = {5},
abstract = {Infrared and visible image fusion (IVIF) aims to provide informative images by combining complementary information from different sensors. Existing IVIF methods based on deep learning focus on strengthening the network with increasing depth but often ignore the importance of transmission characteristics, resulting in the degradation of important information. In addition, while many methods use various loss functions or fusion rules to retain complementary features of both modes, the fusion results often retain redundant or even invalid information.In order to accurately extract the effective information from both infrared images and visible light images without omission or redundancy, and to better serve downstream tasks such as target detection with the fused image, we propose a multi-level structure search attention fusion network based on semantic information guidance, which realizes the fusion of infrared and visible images in an end-to-end way. Our network has two main contributions: the use of neural architecture search (NAS) and the newly designed multilevel adaptive attention module (MAAB). These methods enable our network to retain the typical characteristics of the two modes while removing useless information for the detection task in the fusion results. In addition, our loss function and joint training method can establish a reliable relationship between the fusion network and subsequent detection tasks. Extensive experiments on the new dataset (M3FD) show that our fusion method has achieved advanced performance in both subjective and objective evaluations, and the mAP in the object detection task is improved by 0.5% compared to the second-best method (FusionGAN).},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Infrared and visible image fusion (IVIF) aims to provide informative images by combining complementary information from different sensors. Existing IVIF methods based on deep learning focus on strengthening the network with increasing depth but often ignore the importance of transmission characteristics, resulting in the degradation of important information. In addition, while many methods use various loss functions or fusion rules to retain complementary features of both modes, the fusion results often retain redundant or even invalid information.In order to accurately extract the effective information from both infrared images and visible light images without omission or redundancy, and to better serve downstream tasks such as target detection with the fused image, we propose a multi-level structure search attention fusion network based on semantic information guidance, which realizes the fusion of infrared and visible images in an end-to-end way. Our network has two main contributions: the use of neural architecture search (NAS) and the newly designed multilevel adaptive attention module (MAAB). These methods enable our network to retain the typical characteristics of the two modes while removing useless information for the detection task in the fusion results. In addition, our loss function and joint training method can establish a reliable relationship between the fusion network and subsequent detection tasks. Extensive experiments on the new dataset (M3FD) show that our fusion method has achieved advanced performance in both subjective and objective evaluations, and the mAP in the object detection task is improved by 0.5% compared to the second-best method (FusionGAN).
2761.
Gao, Jianliang; Chen, Jiamin; Oloulade, Babatounde Moctard; Al-Sabri, Raeed; Lyu, Tengfei; Zhang, Ji; Li, Zhao
CommGNAS: Unsupervised Graph Neural Architecture Search for Community Detection Journal Article
In: IEEE Transactions on Emerging Topics in Computing, pp. 1-12, 2023.
@article{10112632,
title = {CommGNAS: Unsupervised Graph Neural Architecture Search for Community Detection},
author = {Jianliang Gao and Jiamin Chen and Babatounde Moctard Oloulade and Raeed Al-Sabri and Tengfei Lyu and Ji Zhang and Zhao Li},
doi = {10.1109/TETC.2023.3270181},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Emerging Topics in Computing},
pages = {1-12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2760.
Bria, Alessandro; Ciccio, Paolo De; DÁlessandro, Tiziana; Fontanella, Francesco
Ä Novel Evolutionary Approach for Neural Architecture Search Proceedings Article
In: Stefano, Claudio De; Fontanella, Francesco; Vanneschi, Leonardo (Ed.): Ärtificial Life and Evolutionary Computation", pp. 195–204, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-31183-3.
@inproceedings{10.1007/978-3-031-31183-3_16,
title = {Ä Novel Evolutionary Approach for Neural Architecture Search},
author = {Alessandro Bria and Paolo De Ciccio and Tiziana DÁlessandro and Francesco Fontanella},
editor = {Claudio De Stefano and Francesco Fontanella and Leonardo Vanneschi},
url = {https://link.springer.com/chapter/10.1007/978-3-031-31183-3_16},
isbn = {978-3-031-31183-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ärtificial Life and Evolutionary Computation"},
pages = {195--204},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Convolutional Neural Networks (CNNs) have proven to be an effective tool in many real-world applications. The main problem of CNNs is the lack of a well-defined and largely shared set of criteria for the choice of architecture for a given problem. This lack represents a drawback for this approach since the choice of architecture plays a crucial role in CNNs'performance. Usually, these architectures are manually designed by experts. However, such a design process is computationally intensive because of the trial-and-error process and also not easy to realize due to the high level of expertise required. Recently, to try to overcome those drawbacks, many techniques that automize the task of designing the architecture neural networks have been proposed. To denote these techniques has been defined the term ``Neural Architecture Search'' (NAS). Among the many methods available for NAS, Evolutionary Computation (EC) methods have recently gained much attention and success. In this paper, we present a novel approach based on evolutionary computation to optimize CNNs. The proposed approach is based on a newly devised structure which encodes both hyperparameters and the architecture of a CNN. The experimental results show that the proposed approach allows us to achieve better performance than that achieved by state-of-the-art CNNs on a real-world problem. Furthermore, the proposed approach can generate smaller networks than the state-of-the-art CNNs used for the comparison.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Convolutional Neural Networks (CNNs) have proven to be an effective tool in many real-world applications. The main problem of CNNs is the lack of a well-defined and largely shared set of criteria for the choice of architecture for a given problem. This lack represents a drawback for this approach since the choice of architecture plays a crucial role in CNNs'performance. Usually, these architectures are manually designed by experts. However, such a design process is computationally intensive because of the trial-and-error process and also not easy to realize due to the high level of expertise required. Recently, to try to overcome those drawbacks, many techniques that automize the task of designing the architecture neural networks have been proposed. To denote these techniques has been defined the term ``Neural Architecture Search'' (NAS). Among the many methods available for NAS, Evolutionary Computation (EC) methods have recently gained much attention and success. In this paper, we present a novel approach based on evolutionary computation to optimize CNNs. The proposed approach is based on a newly devised structure which encodes both hyperparameters and the architecture of a CNN. The experimental results show that the proposed approach allows us to achieve better performance than that achieved by state-of-the-art CNNs on a real-world problem. Furthermore, the proposed approach can generate smaller networks than the state-of-the-art CNNs used for the comparison.
2759.
Ao, Lei; Feng, Kaiyuan; Sheng, Kai; Zhao, Hongyu; He, Xin; Chen, Zigang
TPENAS: A Two-Phase Evolutionary Neural Architecture Search for Remote Sensing Image Classification Journal Article
In: Remote Sensing, vol. 15, no. 8, 2023, ISSN: 2072-4292.
@article{rs15082212,
title = {TPENAS: A Two-Phase Evolutionary Neural Architecture Search for Remote Sensing Image Classification},
author = {Lei Ao and Kaiyuan Feng and Kai Sheng and Hongyu Zhao and Xin He and Zigang Chen},
url = {https://www.mdpi.com/2072-4292/15/8/2212},
doi = {10.3390/rs15082212},
issn = {2072-4292},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Remote Sensing},
volume = {15},
number = {8},
abstract = {The application of deep learning in remote sensing image classification has been paid more and more attention by industry and academia. However, manually designed remote sensing image classification models based on convolutional neural networks usually require sophisticated expert knowledge. Moreover, it is notoriously difficult to design a model with both high classification accuracy and few parameters. Recently, neural architecture search (NAS) has emerged as an effective method that can greatly reduce the heavy burden of manually designing models. However, it remains a challenge to search for a classification model with high classification accuracy and few parameters in the huge search space. To tackle this challenge, we propose TPENAS, a two-phase evolutionary neural architecture search framework, which optimizes the model using computational intelligence techniques in two search phases. In the first search phase, TPENAS searches for the optimal depth of the model. In the second search phase, TPENAS searches for the structure of the model from the perspective of the whole model. Experiments on three open benchmark datasets demonstrate that our proposed TPENAS outperforms the state-of-the-art baselines in both classification accuracy and reducing parameters.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The application of deep learning in remote sensing image classification has been paid more and more attention by industry and academia. However, manually designed remote sensing image classification models based on convolutional neural networks usually require sophisticated expert knowledge. Moreover, it is notoriously difficult to design a model with both high classification accuracy and few parameters. Recently, neural architecture search (NAS) has emerged as an effective method that can greatly reduce the heavy burden of manually designing models. However, it remains a challenge to search for a classification model with high classification accuracy and few parameters in the huge search space. To tackle this challenge, we propose TPENAS, a two-phase evolutionary neural architecture search framework, which optimizes the model using computational intelligence techniques in two search phases. In the first search phase, TPENAS searches for the optimal depth of the model. In the second search phase, TPENAS searches for the structure of the model from the perspective of the whole model. Experiments on three open benchmark datasets demonstrate that our proposed TPENAS outperforms the state-of-the-art baselines in both classification accuracy and reducing parameters.
2758.
Wei, He; Yang, Yuekui; Wu, Haiyang; Tang, Yangyang; Liu, Meixi; Li, Jianfeng
Automatic Feature Selection By One-Shot Neural Architecture Search In Recommendation Systems Proceedings Article
In: Proceedings of the ACM Web Conference 2023, pp. 1765–1772, Association for Computing Machinery, Austin, TX, USA, 2023, ISBN: 9781450394161.
@inproceedings{10.1145/3543507.3583444,
title = {Automatic Feature Selection By One-Shot Neural Architecture Search In Recommendation Systems},
author = {He Wei and Yuekui Yang and Haiyang Wu and Yangyang Tang and Meixi Liu and Jianfeng Li},
url = {https://doi.org/10.1145/3543507.3583444},
doi = {10.1145/3543507.3583444},
isbn = {9781450394161},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the ACM Web Conference 2023},
pages = {1765–1772},
publisher = {Association for Computing Machinery},
address = {Austin, TX, USA},
series = {WWW '23},
abstract = {Feature selection is crucial in large-scale recommendation system, which can not only reduce the computational cost, but also improve the recommendation efficiency. Most existing works rank the features and then select the top-k ones as the final feature subset. However, they assess feature importance individually and ignore the interrelationship between features. Consequently, multiple features with high relevance may be selected simultaneously, resulting in sub-optimal result. In this work, we solve this problem by proposing an AutoML-based feature selection framework that can automatically search the optimal feature subset. Specifically, we first embed the search space into a weight-sharing Supernet. Then, a two-stage neural architecture search method is employed to evaluate the feature quality. In the first stage, a well-designed sampling method considering feature convergence fairness is applied to train the Supernet. In the second stage, a reinforcement learning method is used to search for the optimal feature subset efficiently. The Experimental results on two real datasets demonstrate the superior performance of new framework over other solutions. Our proposed method obtain significant improvement with a 20% reduction in the amount of features on the Criteo. More validation experiments demonstrate the ability and robustness of the framework.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Feature selection is crucial in large-scale recommendation system, which can not only reduce the computational cost, but also improve the recommendation efficiency. Most existing works rank the features and then select the top-k ones as the final feature subset. However, they assess feature importance individually and ignore the interrelationship between features. Consequently, multiple features with high relevance may be selected simultaneously, resulting in sub-optimal result. In this work, we solve this problem by proposing an AutoML-based feature selection framework that can automatically search the optimal feature subset. Specifically, we first embed the search space into a weight-sharing Supernet. Then, a two-stage neural architecture search method is employed to evaluate the feature quality. In the first stage, a well-designed sampling method considering feature convergence fairness is applied to train the Supernet. In the second stage, a reinforcement learning method is used to search for the optimal feature subset efficiently. The Experimental results on two real datasets demonstrate the superior performance of new framework over other solutions. Our proposed method obtain significant improvement with a 20% reduction in the amount of features on the Criteo. More validation experiments demonstrate the ability and robustness of the framework.
2757.
Wu, Shiguang; Wang, Yaqing; Jing, Qinghe; Dong, Daxiang; Dou, Dejing; Yao, Quanming
ColdNAS: Search to Modulate for User Cold-Start Recommendation Proceedings Article
In: Proceedings of the ACM Web Conference 2023, pp. 1021–1031, Association for Computing Machinery, Austin, TX, USA, 2023, ISBN: 9781450394161.
@inproceedings{10.1145/3543507.3583344,
title = {ColdNAS: Search to Modulate for User Cold-Start Recommendation},
author = {Shiguang Wu and Yaqing Wang and Qinghe Jing and Daxiang Dong and Dejing Dou and Quanming Yao},
url = {https://doi.org/10.1145/3543507.3583344},
doi = {10.1145/3543507.3583344},
isbn = {9781450394161},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the ACM Web Conference 2023},
pages = {1021–1031},
publisher = {Association for Computing Machinery},
address = {Austin, TX, USA},
series = {WWW '23},
abstract = {Making personalized recommendation for cold-start users, who only have a few interaction histories, is a challenging problem in recommendation systems. Recent works leverage hypernetworks to directly map user interaction histories to user-specific parameters, which are then used to modulate predictor by feature-wise linear modulation function. These works obtain the state-of-the-art performance. However, the physical meaning of scaling and shifting in recommendation data is unclear. Instead of using a fixed modulation function and deciding modulation position by expertise, we propose a modulation framework called ColdNAS for user cold-start problem, where we look for proper modulation structure, including function and position, via neural architecture search. We design a search space which covers broad models and theoretically prove that this search space can be transformed to a much smaller space, enabling an efficient and robust one-shot search algorithm. Extensive experimental results on benchmark datasets show that ColdNAS consistently performs the best. We observe that different modulation functions lead to the best performance on different datasets, which validates the necessity of designing a searching-based method. Codes are available at https://github.com/LARS-research/ColdNAS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Making personalized recommendation for cold-start users, who only have a few interaction histories, is a challenging problem in recommendation systems. Recent works leverage hypernetworks to directly map user interaction histories to user-specific parameters, which are then used to modulate predictor by feature-wise linear modulation function. These works obtain the state-of-the-art performance. However, the physical meaning of scaling and shifting in recommendation data is unclear. Instead of using a fixed modulation function and deciding modulation position by expertise, we propose a modulation framework called ColdNAS for user cold-start problem, where we look for proper modulation structure, including function and position, via neural architecture search. We design a search space which covers broad models and theoretically prove that this search space can be transformed to a much smaller space, enabling an efficient and robust one-shot search algorithm. Extensive experimental results on benchmark datasets show that ColdNAS consistently performs the best. We observe that different modulation functions lead to the best performance on different datasets, which validates the necessity of designing a searching-based method. Codes are available at https://github.com/LARS-research/ColdNAS.
2756.
Xie, Xiangning; Sun, Yanan; Liu, Yuqiao; Zhang, Mengjie; Tan, Kay Chen
Architecture Augmentation for Performance Predictor via Graph Isomorphism Journal Article
In: IEEE Transactions on Cybernetics, pp. 1-13, 2023.
@article{10109990,
title = {Architecture Augmentation for Performance Predictor via Graph Isomorphism},
author = {Xiangning Xie and Yanan Sun and Yuqiao Liu and Mengjie Zhang and Kay Chen Tan},
doi = {10.1109/TCYB.2023.3267109},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Cybernetics},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2755.
Mansoori, Mohammad Amir; Casu, Mario R.
Multi-objective Framework for Training and Hardware Co-optimization in FPGAs Proceedings Article
In: Berta, Riccardo; Gloria, Alessandro De (Ed.): Äpplications in Electronics Pervading Industry, Environment and Society", pp. 273–278, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-30333-3.
@inproceedings{10.1007/978-3-031-30333-3_36,
title = {Multi-objective Framework for Training and Hardware Co-optimization in FPGAs},
author = {Mohammad Amir Mansoori and Mario R. Casu},
editor = {Riccardo Berta and Alessandro De Gloria},
url = {https://link.springer.com/chapter/10.1007/978-3-031-30333-3_36},
isbn = {978-3-031-30333-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Äpplications in Electronics Pervading Industry, Environment and Society"},
pages = {273--278},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Älthough several works have recently addressed the problem of performance co-optimization for hardware and network training for Convolutional Neural Networks, most of them considered either a fixed network or a given hardware architecture. In this work, we propose a new framework for joint optimization of network architecture and hardware configurations based on Bayesian Optimization (BO) on top of High Level Synthesis. The multi-objective nature of this framework allows for the definition of various hardware and network performance goals as well as multiple constraints, and the multi-objective BO allows to easily obtain a set of Pareto points. We evaluate our methodology on a network optimized for an FPGA target and show that the Pareto set obtained by the proposed joint-optimization outperforms other methods based on a separate optimization or random search."},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Älthough several works have recently addressed the problem of performance co-optimization for hardware and network training for Convolutional Neural Networks, most of them considered either a fixed network or a given hardware architecture. In this work, we propose a new framework for joint optimization of network architecture and hardware configurations based on Bayesian Optimization (BO) on top of High Level Synthesis. The multi-objective nature of this framework allows for the definition of various hardware and network performance goals as well as multiple constraints, and the multi-objective BO allows to easily obtain a set of Pareto points. We evaluate our methodology on a network optimized for an FPGA target and show that the Pareto set obtained by the proposed joint-optimization outperforms other methods based on a separate optimization or random search."
2754.
Guo, Xinwei; Wu, Yong; Miao, Jingjing; Chen, Yang
LiteGaze: Neural architecture search for efficient gaze estimation Journal Article
In: PLOS ONE, vol. 18, no. 5, pp. 1-11, 2023.
@article{10.1371/journal.pone.0284814,
title = {LiteGaze: Neural architecture search for efficient gaze estimation},
author = {Xinwei Guo and Yong Wu and Jingjing Miao and Yang Chen},
url = {https://doi.org/10.1371/journal.pone.0284814},
doi = {10.1371/journal.pone.0284814},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {PLOS ONE},
volume = {18},
number = {5},
pages = {1-11},
publisher = {Public Library of Science},
abstract = {Gaze estimation plays a critical role in human-centered vision applications such as human–computer interaction and virtual reality. Although significant progress has been made in automatic gaze estimation by deep convolutional neural networks, it is still difficult to directly deploy deep learning based gaze estimation models across different edge devices, due to the high computational cost and various resource constraints. This work proposes LiteGaze, a deep learning framework to learn architectures for efficient gaze estimation via neural architecture search (NAS). Inspired by the once-for-all model (Cai et al., 2020), this work decouples the model training and architecture search into two different stages. In particular, a supernet is trained to support diverse architectural settings. Then specialized sub-networks are selected from the obtained supernet, given different efficiency constraints. Extensive experiments are performed on two gaze estimation datasets and demonstrate the superiority of the proposed method over previous works, advancing the real-time gaze estimation on edge devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Gaze estimation plays a critical role in human-centered vision applications such as human–computer interaction and virtual reality. Although significant progress has been made in automatic gaze estimation by deep convolutional neural networks, it is still difficult to directly deploy deep learning based gaze estimation models across different edge devices, due to the high computational cost and various resource constraints. This work proposes LiteGaze, a deep learning framework to learn architectures for efficient gaze estimation via neural architecture search (NAS). Inspired by the once-for-all model (Cai et al., 2020), this work decouples the model training and architecture search into two different stages. In particular, a supernet is trained to support diverse architectural settings. Then specialized sub-networks are selected from the obtained supernet, given different efficiency constraints. Extensive experiments are performed on two gaze estimation datasets and demonstrate the superiority of the proposed method over previous works, advancing the real-time gaze estimation on edge devices.
2753.
Wang, Haibin; Ge, Ce; Chen, Hesen; Sun, Xiuyu
PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-14636,
title = {PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search},
author = {Haibin Wang and Ce Ge and Hesen Chen and Xiuyu Sun},
url = {https://doi.org/10.48550/arXiv.2304.14636},
doi = {10.48550/arXiv.2304.14636},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.14636},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2752.
Singh, Bhavneet; Mansukhani, Jai
Comparing Neural Architectures to Find the Best Model Suited for Edge Devices Proceedings Article
In: Sharma, Harish; Saha, Apu Kumar; Prasad, Mukesh (Ed.): Proceedings of International Conference on Intelligent Vision and Computing (ICIVC 2022), pp. 195–203, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-31164-2.
@inproceedings{10.1007/978-3-031-31164-2_16,
title = {Comparing Neural Architectures to Find the Best Model Suited for Edge Devices},
author = {Bhavneet Singh and Jai Mansukhani},
editor = {Harish Sharma and Apu Kumar Saha and Mukesh Prasad},
url = {https://link.springer.com/chapter/10.1007/978-3-031-31164-2_16},
isbn = {978-3-031-31164-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of International Conference on Intelligent Vision and Computing (ICIVC 2022)},
pages = {195--203},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Training large-scale neural network models is computationally expensive and demands a great deal of resources. It is an important area of study with a lot of potential for the future of the AI industry. In recent years, the power of computer hardware has significantly improved and we have new breakthroughs in deep learning. With these innovations, the computational cost of training large neural network models has declined by at least 10 folds in high- and average-performance machines. In this research, we explore NAL, AutoML, and other frameworks to determine the best suitable model for edge devices. The biggest improvements compared to reference models can be acquired if the NAS algorithm is co-designed with the corresponding inference engine.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Training large-scale neural network models is computationally expensive and demands a great deal of resources. It is an important area of study with a lot of potential for the future of the AI industry. In recent years, the power of computer hardware has significantly improved and we have new breakthroughs in deep learning. With these innovations, the computational cost of training large neural network models has declined by at least 10 folds in high- and average-performance machines. In this research, we explore NAL, AutoML, and other frameworks to determine the best suitable model for edge devices. The biggest improvements compared to reference models can be acquired if the NAS algorithm is co-designed with the corresponding inference engine.
2751.
Mlodozeniec, Bruno; Reisser, Matthias; Louizos, Christos
Hyperparameter Optimization through Neural Network Partitioning Proceedings Article
In: ICLR 2023, 2023.
@inproceedings{DBLP:journals/corr/abs-2304-14766,
title = {Hyperparameter Optimization through Neural Network Partitioning},
author = {Bruno Mlodozeniec and Matthias Reisser and Christos Louizos},
url = {https://doi.org/10.48550/arXiv.2304.14766},
doi = {10.48550/arXiv.2304.14766},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = { ICLR 2023},
journal = { ICLR 2023},
volume = {abs/2304.14766},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2750.
Garcia-Garcia, Cosijopii; Morales-Reyes, Alicia; Escalante, Hugo
Continuous Cartesian Genetic Programming based representation for Multi-Objective Neural Architecture Search Technical Report
2023.
@techreport{unknown,
title = {Continuous Cartesian Genetic Programming based representation for Multi-Objective Neural Architecture Search},
author = {Cosijopii Garcia-Garcia and Alicia Morales-Reyes and Hugo Escalante},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2749.
Chen, Yu; Yang, Mingyu; Kim, Hun-Seok
Search for Efficient Deep Visual-Inertial Odometry Through Neural Architecture Search Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10095166,
title = {Search for Efficient Deep Visual-Inertial Odometry Through Neural Architecture Search},
author = {Yu Chen and Mingyu Yang and Hun-Seok Kim},
url = {https://ieeexplore.ieee.org/abstract/document/10095166},
doi = {10.1109/ICASSP49357.2023.10095166},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2748.
Keserwani, Prateek; Miriyala, Srinivas Soumitri; Rajendiran, Vikram N.; Shivamurthappa, Pradeep N.
Receptive Field Reliant Zero-Cost Proxies for Neural Architecture Search Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10096753,
title = {Receptive Field Reliant Zero-Cost Proxies for Neural Architecture Search},
author = {Prateek Keserwani and Srinivas Soumitri Miriyala and Vikram N. Rajendiran and Pradeep N. Shivamurthappa},
url = {https://ieeexplore.ieee.org/abstract/document/10096621},
doi = {10.1109/ICASSP49357.2023.10096753},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2747.
Rumiantsev, Pavel; Coates, Mark
Performing Neural Architecture Search Without Gradients Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10094582,
title = {Performing Neural Architecture Search Without Gradients},
author = {Pavel Rumiantsev and Mark Coates},
url = {https://ieeexplore.ieee.org/abstract/document/10094582},
doi = {10.1109/ICASSP49357.2023.10094582},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2746.
Fan, Yi; Niu, Zhong-Han; Yang, Yu-Bin
Data-Aware Zero-Shot Neural Architecture Search for Image Recognition Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10094741,
title = {Data-Aware Zero-Shot Neural Architecture Search for Image Recognition},
author = {Yi Fan and Zhong-Han Niu and Yu-Bin Yang},
url = {https://ieeexplore.ieee.org/abstract/document/10094741},
doi = {10.1109/ICASSP49357.2023.10094741},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2745.
Xie, Lunchen; Huang, Kaiyu; Xu, Fan; Shi, Qingjiang
ZO-DARTS: Differentiable Architecture Search with Zeroth-Order Approximation Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10096612,
title = {ZO-DARTS: Differentiable Architecture Search with Zeroth-Order Approximation},
author = {Lunchen Xie and Kaiyu Huang and Fan Xu and Qingjiang Shi},
url = {https://ieeexplore.ieee.org/abstract/document/10096612},
doi = {10.1109/ICASSP49357.2023.10096612},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2744.
Cai, Zicheng; Chen, Lei; Liu, Hai-Lin
BHE-DARTS: Bilevel Optimization Based on Hypergradient Estimation for Differentiable Architecture Search Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10095940,
title = {BHE-DARTS: Bilevel Optimization Based on Hypergradient Estimation for Differentiable Architecture Search},
author = {Zicheng Cai and Lei Chen and Hai-Lin Liu},
url = {https://ieeexplore.ieee.org/abstract/document/10095940},
doi = {10.1109/ICASSP49357.2023.10095940},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2743.
Jeong, Joonhyun; Yu, Joonsang; Park, Geondo; Han, Dongyoon; Yoo, Youngjoon
GeNAS: Neural Architecture Search with Better Generalization Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-08611,
title = {GeNAS: Neural Architecture Search with Better Generalization},
author = {Joonhyun Jeong and Joonsang Yu and Geondo Park and Dongyoon Han and Youngjoon Yoo},
url = {https://doi.org/10.48550/arXiv.2305.08611},
doi = {10.48550/arXiv.2305.08611},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.08611},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2742.
Zhang, Jianwei; Zhang, Lei; Li, Dong
A Unified Search Framework for Data Augmentation and Neural Architecture on Small-scale Image Datasets Journal Article
In: IEEE Transactions on Cognitive and Developmental Systems, pp. 1-1, 2023.
@article{10124814,
title = {A Unified Search Framework for Data Augmentation and Neural Architecture on Small-scale Image Datasets},
author = {Jianwei Zhang and Lei Zhang and Dong Li},
url = {https://ieeexplore.ieee.org/abstract/document/10124814},
doi = {10.1109/TCDS.2023.3274177},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Cognitive and Developmental Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2741.
Sun, Jialiang; Yao, Wen; Jiang, Tingsong; Chen, Xiaoqian
Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-07308,
title = {Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation},
author = {Jialiang Sun and Wen Yao and Tingsong Jiang and Xiaoqian Chen},
url = {https://doi.org/10.48550/arXiv.2305.07308},
doi = {10.48550/arXiv.2305.07308},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.07308},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2740.
Li, Guihong; Bhardwaj, Kartikeya; Yang, Yuedong; Marculescu, Radu
TIPS: Topologically Important Path Sampling for Anytime Neural Networks Journal Article
In: CoRR, vol. abs/2305.08021, 2023.
@article{DBLP:journals/corr/abs-2305-08021,
title = {TIPS: Topologically Important Path Sampling for Anytime Neural Networks},
author = {Guihong Li and Kartikeya Bhardwaj and Yuedong Yang and Radu Marculescu},
url = {https://doi.org/10.48550/arXiv.2305.08021},
doi = {10.48550/arXiv.2305.08021},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.08021},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2739.
Hadi, Russul H.; Hady, Haider N.; Hasan, Ahmed M.; Al-Jodah, Ammar; Humaidi, Amjad J.
Improved Fault Classification for Predictive Maintenance in Industrial IoT Based on AutoML: A Case Study of Ball-Bearing Faults Journal Article
In: Processes, vol. 11, no. 5, 2023, ISSN: 2227-9717.
@article{pr11051507,
title = {Improved Fault Classification for Predictive Maintenance in Industrial IoT Based on AutoML: A Case Study of Ball-Bearing Faults},
author = {Russul H. Hadi and Haider N. Hady and Ahmed M. Hasan and Ammar Al-Jodah and Amjad J. Humaidi},
url = {https://www.mdpi.com/2227-9717/11/5/1507},
doi = {10.3390/pr11051507},
issn = {2227-9717},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Processes},
volume = {11},
number = {5},
abstract = {The growing complexity of data derived from Industrial Internet of Things (IIoT) systems presents substantial challenges for traditional machine-learning techniques, which struggle to effectively manage the needs of predictive maintenance applications. Automated machine-learning (AutoML) techniques present a promising solution by streamlining the machine-learning process, reducing the necessity for manual hyperparameter tuning and computational resources, thereby positioning themselves as a potentially transformative innovation in the Industry 4.0 era. This research introduces two distinct models: AutoML, employing PyCaret, and Auto Deep Neural Network (AutoDNN), utilizing AutoKeras, both aimed at accurately identifying various types of faults in ball bearings. The proposed models were evaluated using the Case Western Reserve University (CWRU) bearing faults dataset, and the results showed a notable performance in terms of achieving high accuracy, recall, precision, and F1 score on the testing and validation sets. Compared to recent studies, the proposed AutoML models demonstrated superior performance, surpassing alternative approaches even when they utilized a larger number of features, thus highlighting the effectiveness of the proposed methodology. This research offers valuable insights for those interested in harnessing the potential of AutoML techniques in IIoT applications, with implications for industries such as manufacturing and energy. By automating the machine-learning process, AutoML models can help decrease the time and cost related to predictive maintenance, which is crucial for industries where unplanned downtime can lead to substantial financial losses.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The growing complexity of data derived from Industrial Internet of Things (IIoT) systems presents substantial challenges for traditional machine-learning techniques, which struggle to effectively manage the needs of predictive maintenance applications. Automated machine-learning (AutoML) techniques present a promising solution by streamlining the machine-learning process, reducing the necessity for manual hyperparameter tuning and computational resources, thereby positioning themselves as a potentially transformative innovation in the Industry 4.0 era. This research introduces two distinct models: AutoML, employing PyCaret, and Auto Deep Neural Network (AutoDNN), utilizing AutoKeras, both aimed at accurately identifying various types of faults in ball bearings. The proposed models were evaluated using the Case Western Reserve University (CWRU) bearing faults dataset, and the results showed a notable performance in terms of achieving high accuracy, recall, precision, and F1 score on the testing and validation sets. Compared to recent studies, the proposed AutoML models demonstrated superior performance, surpassing alternative approaches even when they utilized a larger number of features, thus highlighting the effectiveness of the proposed methodology. This research offers valuable insights for those interested in harnessing the potential of AutoML techniques in IIoT applications, with implications for industries such as manufacturing and energy. By automating the machine-learning process, AutoML models can help decrease the time and cost related to predictive maintenance, which is crucial for industries where unplanned downtime can lead to substantial financial losses.
2738.
Wang, Yifan; Zhong, Linlin
NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs Technical Report
2023.
@techreport{wang2023naspinn,
title = {NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs},
author = {Yifan Wang and Linlin Zhong},
url = {https://arxiv.org/abs/2305.10127},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2737.
Benmeziane, Hadjer; Lammie, Corey; Boybat, Irem; Rasch, Malte; Gallo, Manuel Le; Tsai, Hsinyu; Muralidhar, Ramachandran; Niar, Smail; Hamza, Ouarnoughi; Narayanan, Vijay; Sebastian, Abu; Maghraoui, Kaoutar El
AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing Technical Report
2023.
@techreport{benmeziane2023analognas,
title = {AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing},
author = {Hadjer Benmeziane and Corey Lammie and Irem Boybat and Malte Rasch and Manuel Le Gallo and Hsinyu Tsai and Ramachandran Muralidhar and Smail Niar and Ouarnoughi Hamza and Vijay Narayanan and Abu Sebastian and Kaoutar El Maghraoui},
url = {https://arxiv.org/abs/2305.10459},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}