Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2024
3438.
Xu, Hongshang; Dong, Bei; Liu, Xiaochang; Wu, Xiaojun
Deep Neural Network Architecture Search via Decomposition-Based Multi-Objective Stochastic Fractal Search Proceedings Article
In: Intelligent Automation & Soft Computing, 2024.
@inproceedings{Xu-IASC24a,
title = {Deep Neural Network Architecture Search via Decomposition-Based Multi-Objective Stochastic Fractal Search},
author = {Hongshang Xu and Bei Dong and Xiaochang Liu and Xiaojun Wu},
url = {https://cdn.techscience.cn/files/iasc/2023/TSP_IASC-38-2/TSP_IASC_41177/TSP_IASC_41177.pdf},
doi = {10.32604/iasc.2023.041177},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
booktitle = {Intelligent Automation & Soft Computing},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3437.
Dou, Huanzhang; Zhang, Pengyi; Zhao, Yuhan; Jin, Lu; Li, Xi
CLASH: Complementary Learning with Neural Architecture Search for Gait Recognition Bachelor Thesis
2024.
@bachelorthesis{Dou-tip24a,
title = { CLASH: Complementary Learning with Neural Architecture Search for Gait Recognition },
author = {Huanzhang Dou and Pengyi Zhang and Yuhan Zhao and Lu Jin and Xi Li
},
url = {https://pubmed.ncbi.nlm.nih.gov/38363666/},
doi = { 10.1109/TIP.2024.3360870 },
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = { IEEE Trans Image Process },
keywords = {},
pubstate = {published},
tppubtype = {bachelorthesis}
}
3436.
Ma, L.; Kang, H.; Yu, G.; Li, Q.; He, Q.
Single-Domain Generalized Predictor for Neural Architecture Search System Journal Article
In: IEEE Transactions on Computers, no. 01, pp. 1-14, 2024, ISSN: 1557-9956.
@article{10438213,
title = {Single-Domain Generalized Predictor for Neural Architecture Search System},
author = {L. Ma and H. Kang and G. Yu and Q. Li and Q. He},
doi = {10.1109/TC.2024.3365949},
issn = {1557-9956},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = {IEEE Transactions on Computers},
number = {01},
pages = {1-14},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Performance predictors are used to reduce architecture evaluation costs in neural architecture search, which however suffers from a large amount of budget consumption in annotating substantial architectures trained from scratch. Hence, how to leverage existing annotated architectures to train a generalized predictor to find the optimal architecture on unseen target search spaces becomes a new research topic. To solve this issue, we propose a Single-Domain Generalized Predictor (SDGP), which aims to make the predictor only trained on a single source search space but perform well on target search spaces. In meta-learning, we firstly adopt feature extractor in learning the domain-invariant features of the architectures. Then, a neural predictor is trained to map the architectures to the accuracy of the candidate architectures over the target domain simulated on the source search space. Moreover, a novel multi-head attention driven regularizer is designed to regulate the predictor to further improve the generalization ability of the predictor for the feature extractor. A series of experimental results have shown that the proposed predictor outperforms the state-of-the-art predictors in generalization and achieves significant performance gains in finding the optimal architectures with test error 2.40% on CIFAR-10 and 23.20% on ImageNet1k within 0.01 GPU days.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Performance predictors are used to reduce architecture evaluation costs in neural architecture search, which however suffers from a large amount of budget consumption in annotating substantial architectures trained from scratch. Hence, how to leverage existing annotated architectures to train a generalized predictor to find the optimal architecture on unseen target search spaces becomes a new research topic. To solve this issue, we propose a Single-Domain Generalized Predictor (SDGP), which aims to make the predictor only trained on a single source search space but perform well on target search spaces. In meta-learning, we firstly adopt feature extractor in learning the domain-invariant features of the architectures. Then, a neural predictor is trained to map the architectures to the accuracy of the candidate architectures over the target domain simulated on the source search space. Moreover, a novel multi-head attention driven regularizer is designed to regulate the predictor to further improve the generalization ability of the predictor for the feature extractor. A series of experimental results have shown that the proposed predictor outperforms the state-of-the-art predictors in generalization and achieves significant performance gains in finding the optimal architectures with test error 2.40% on CIFAR-10 and 23.20% on ImageNet1k within 0.01 GPU days.
3435.
Gong, Tao; Ma, Yongjie; Xu, Yang; Song, Changwei
Efficient evolutionary neural architecture search based on hybrid search space Journal Article
In: International Journal of Machine Learning and Cybernetics, 2024.
@article{Gong-ijmlc24a,
title = {Efficient evolutionary neural architecture search based on hybrid search space},
author = {Tao Gong and Yongjie Ma and Yang Xu and Changwei Song},
url = {https://link.springer.com/article/10.1007/s13042-023-02094-z},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = {International Journal of Machine Learning and Cybernetics},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3434.
(Ed.)
Personalized Federated Learning via Knowledge Sharing-based Model Structure Adaption Collection
2024.
@collection{wang-ijcnn23a,
title = {Personalized Federated Learning via Knowledge Sharing-based Model Structure Adaption},
author = {Xiaochan Wang and Zhi Wang},
url = {http://zwang.inflexionlab.org/publications/FedKMA_IJCNN2023.pdf},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
booktitle = {IJCNN 2023},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3433.
Bouali, Yassamine Lala; Ahmed, Olfa Ben; Bradai, Abbas; Mazouzi, Smaine
Towards Efficient Driver Distraction Detection with DARTS-Optimized Lightweight Models Proceedings Article
In: ICAART 2024, Rome, Italy, 2024.
@inproceedings{lalabouali:hal-04447892,
title = {Towards Efficient Driver Distraction Detection with DARTS-Optimized Lightweight Models},
author = {Yassamine Lala Bouali and Olfa Ben Ahmed and Abbas Bradai and Smaine Mazouzi},
url = {https://hal.science/hal-04447892},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
booktitle = {ICAART 2024},
address = {Rome, Italy},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3432.
Kumar, Santhosh; Sasirekha, S. P.; Santhosh, R.
LSTM-NAS-Net: Enhanced Brain Tumor Segmentation in MRI and CT Images using LSTM-Autoencoder-based Neural Architecture Search Journal Article
In: Journal of Cybersecurity and Information Management, vol. 14, iss. 2, 2024.
@article{kumar-jcim24a,
title = { LSTM-NAS-Net: Enhanced Brain Tumor Segmentation in MRI and CT Images using LSTM-Autoencoder-based Neural Architecture Search },
author = { Santhosh Kumar and S. P. Sasirekha and R. Santhosh },
url = {https://www.americaspg.com/articleinfo/2/show/2965},
year = {2024},
date = {2024-01-17},
urldate = {2024-01-17},
journal = {Journal of Cybersecurity and Information Management},
volume = {14},
issue = {2},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3431.
Yang, Yi; Wei, Jiaxuan; Yu, Yhixuan; Zhang, Ruisheng
Multi-label neural architecture search for chest radiography image classification Journal Article
In: Multimedia Systems, 2024.
@article{nokey,
title = {Multi-label neural architecture search for chest radiography image classification},
author = {Yi Yang and Jiaxuan Wei and Yhixuan Yu and Ruisheng Zhang},
url = {https://link.springer.com/article/10.1007/s00530-023-01215-6},
year = {2024},
date = {2024-01-13},
urldate = {2024-01-13},
journal = { Multimedia Systems},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3430.
LI, PEIXIA
Deep Neural Networks for Visual Object Tracking: An Investigation of Performance Optimization PhD Thesis
2024.
@phdthesis{li-phd23a,
title = {Deep Neural Networks for Visual Object Tracking: An Investigation of Performance Optimization},
author = {PEIXIA LI},
url = {https://scholar.google.de/scholar_url?url=https://ses.library.usyd.edu.au/bitstream/handle/2123/32052/li_p_thesis.pdf%3Fsequence%3D1%26isAllowed%3Dy&hl=de&sa=X&d=18223269356716447880&ei=EsGfZemVKsKAy9YP4bqs-Ac&scisig=AFWwaeYD2Xl7kLWLITGKadrDAlTa&oi=scholaralrt&hist=mvciDDAAAAAJ:2945779489622371749:AFWwaeYSwjSBxI9k5p1JRsFqGwve&html=&pos=2&folt=kw},
year = {2024},
date = {2024-01-12},
urldate = {2024-01-12},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3429.
Dong, Minjing
Boosting Adversarial Robustness via Neural Architecture Search and Design PhD Thesis
2024.
@phdthesis{dong-phd23a,
title = {Boosting Adversarial Robustness via Neural Architecture Search and Design},
author = {Minjing Dong },
url = {https://ses.library.usyd.edu.au/bitstream/handle/2123/32060/dong_md_thesis.pdf?sequence=1&isAllowed=y},
year = {2024},
date = {2024-01-12},
urldate = {2024-01-12},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3428.
(Ed.)
ENHANCING GAN PERFORMANCE THROUGH NEURAL ARCHITECTURE SEARCH AND TENSOR DECOMPOSITION Collection
2024.
@collection{Pulakurthi-icassp24a,
title = {ENHANCING GAN PERFORMANCE THROUGH NEURAL ARCHITECTURE SEARCH AND TENSOR DECOMPOSITION},
author = {Prasanna Reddy Pulakurthi and Mahsa Mozaffari and Majid Rabbani and Jamison Heard and Raghuveer Rao},
url = {https://mahsamozaffari.com/wp-content/uploads/2023/11/ICASSP_2024.pdf},
year = {2024},
date = {2024-01-02},
urldate = {2024-01-02},
booktitle = {ICASSP 2024},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3427.
Liao, Peng; Wang, XiLu; Jin, Yaochu; Du, WenLi
MO-EMT-NAS: Multi-Objective Continuous Transfer of Architectural Knowledge Between Tasks from Different Datasets Technical Report
2024.
@techreport{liao2024moemtnasmultiobjectivecontinuoustransfer,
title = {MO-EMT-NAS: Multi-Objective Continuous Transfer of Architectural Knowledge Between Tasks from Different Datasets},
author = {Peng Liao and XiLu Wang and Yaochu Jin and WenLi Du},
url = {https://arxiv.org/abs/2407.13122},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3426.
Zaitoon, Ruqsar; Mohanty, Sachi Nandan; Godavarthi, Deepthi; Ramesh, J. V. N.
In: IEEE Access, pp. 1-1, 2024.
@article{10600675,
title = {SPBTGNS: Design of an Efficient Model for Survival Prediction in Brain Tumour Patients using Generative Adversarial Network with Neural Architectural Search Operations},
author = {Ruqsar Zaitoon and Sachi Nandan Mohanty and Deepthi Godavarthi and J. V. N. Ramesh},
url = {https://ieeexplore.ieee.org/document/10600675},
doi = {10.1109/ACCESS.2024.3430074},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {IEEE Access},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3425.
Ye, Songtao; Zheng, Saisai; Xia, Yizhang
Channel Attention-Based Method for Searching Task-Specific Multi-Task Network Structures Proceedings Article
In: 2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp. 562-569, 2024.
@inproceedings{10580580,
title = {Channel Attention-Based Method for Searching Task-Specific Multi-Task Network Structures},
author = {Songtao Ye and Saisai Zheng and Yizhang Xia},
doi = {10.1109/CSCWD61410.2024.10580580},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)},
pages = {562-569},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3424.
Yang, Shangshang; Ma, Haiping; Bi, Ying; Tian, Ye; Zhang, Limiao; Jin, Yaochu; Zhang, Xingyi
An Evolutionary Multi-Objective Neural Architecture Search Approach to Advancing Cognitive Diagnosis in Intelligent Education Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2024.
@article{10599558,
title = {An Evolutionary Multi-Objective Neural Architecture Search Approach to Advancing Cognitive Diagnosis in Intelligent Education},
author = {Shangshang Yang and Haiping Ma and Ying Bi and Ye Tian and Limiao Zhang and Yaochu Jin and Xingyi Zhang},
url = {https://ieeexplore.ieee.org/abstract/document/10599558},
doi = {10.1109/TEVC.2024.3429180},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3423.
Li, Zhikai; Long, Xianlei; Xiao, Junrui; Gu, Qingyi
HTQ: Exploring the High-Dimensional Trade-Off of mixed-precision quantization Journal Article
In: Pattern Recognition, vol. 156, pp. 110788, 2024, ISSN: 0031-3203.
@article{LI2024110788,
title = {HTQ: Exploring the High-Dimensional Trade-Off of mixed-precision quantization},
author = {Zhikai Li and Xianlei Long and Junrui Xiao and Qingyi Gu},
url = {https://www.sciencedirect.com/science/article/pii/S0031320324005399},
doi = {https://doi.org/10.1016/j.patcog.2024.110788},
issn = {0031-3203},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Pattern Recognition},
volume = {156},
pages = {110788},
abstract = {Mixed-precision quantization, where more sensitive layers are kept at higher precision, can achieve the trade-off between accuracy and complexity of neural networks. However, the search space for mixed-precision grows exponentially with the number of layers, making the brute force approach infeasible on deep networks. To reduce this exponential search space, recent efforts use Pareto frontier or integer linear programming to select the bit-precision of each layer. Unfortunately, we find that these prior works rely on a single constraint. In practice, model complexity includes space complexity and time complexity, and the two are weakly correlated, thus using simply one as a constraint leads to sub-optimal results. Besides this, they require manually set constraints, making them only pseudo-automatic. To address the above issues, we propose High-dimensional Trade-off Quantization (HTQ), which automatically determines the bit-precision in the high-dimensional space of model accuracy, space complexity, and time complexity without any manual intervention. Specifically, we use the saliency criterion based on connection sensitivity to indicate the accuracy perturbation after quantization, which performs similarly to Hessian information but can be calculated quickly (more than 1000× speedup). The bit-precision is then automatically selected according to the three-dimensional (3D) Pareto frontier of the total perturbation, model size, and bit operations (BOPs) without manual constraints. Moreover, HTQ allows for the joint optimization of weights and activations, and thus the bit-precisions of both can be computed concurrently. Compared to state-of-the-art methods, HTQ achieves higher accuracy and lower space/time complexity on various model architectures for image classification and object detection tasks. Code is available at: https://github.com/zkkli/HTQ.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Mixed-precision quantization, where more sensitive layers are kept at higher precision, can achieve the trade-off between accuracy and complexity of neural networks. However, the search space for mixed-precision grows exponentially with the number of layers, making the brute force approach infeasible on deep networks. To reduce this exponential search space, recent efforts use Pareto frontier or integer linear programming to select the bit-precision of each layer. Unfortunately, we find that these prior works rely on a single constraint. In practice, model complexity includes space complexity and time complexity, and the two are weakly correlated, thus using simply one as a constraint leads to sub-optimal results. Besides this, they require manually set constraints, making them only pseudo-automatic. To address the above issues, we propose High-dimensional Trade-off Quantization (HTQ), which automatically determines the bit-precision in the high-dimensional space of model accuracy, space complexity, and time complexity without any manual intervention. Specifically, we use the saliency criterion based on connection sensitivity to indicate the accuracy perturbation after quantization, which performs similarly to Hessian information but can be calculated quickly (more than 1000× speedup). The bit-precision is then automatically selected according to the three-dimensional (3D) Pareto frontier of the total perturbation, model size, and bit operations (BOPs) without manual constraints. Moreover, HTQ allows for the joint optimization of weights and activations, and thus the bit-precisions of both can be computed concurrently. Compared to state-of-the-art methods, HTQ achieves higher accuracy and lower space/time complexity on various model architectures for image classification and object detection tasks. Code is available at: https://github.com/zkkli/HTQ.
3422.
Li, Zhuojin; Paolieri, Marco; Golubchik, Leana
Inference latency prediction for CNNs on heterogeneous mobile devices and ML frameworks Journal Article
In: Performance Evaluation, vol. 165, pp. 102429, 2024, ISSN: 0166-5316.
@article{LI2024102429,
title = {Inference latency prediction for CNNs on heterogeneous mobile devices and ML frameworks},
author = {Zhuojin Li and Marco Paolieri and Leana Golubchik},
url = {https://www.sciencedirect.com/science/article/pii/S0166531624000348},
doi = {https://doi.org/10.1016/j.peva.2024.102429},
issn = {0166-5316},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Performance Evaluation},
volume = {165},
pages = {102429},
abstract = {Due to the proliferation of inference tasks on mobile devices, state-of-the-art neural architectures are typically designed using Neural Architecture Search (NAS) to achieve good tradeoffs between machine learning accuracy and inference latency. While measuring inference latency of a huge set of candidate architectures during NAS is not feasible, latency prediction for mobile devices is challenging, because of hardware heterogeneity, optimizations applied by machine learning frameworks, and diversity of neural architectures. Motivated by these challenges, we first quantitatively assess the characteristics of neural architectures (specifically, convolutional neural networks for image classification), ML frameworks, and mobile devices that have significant effects on inference latency. Based on this assessment, we propose an operation-wise framework which addresses these challenges by developing operation-wise latency predictors and achieves high accuracy in end-to-end latency predictions, as shown by our comprehensive evaluations on multiple mobile devices using multicore CPUs and GPUs. To illustrate that our approach does not require expensive data collection, we also show that accurate predictions can be achieved on real-world neural architectures using only small amounts of profiling data.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Due to the proliferation of inference tasks on mobile devices, state-of-the-art neural architectures are typically designed using Neural Architecture Search (NAS) to achieve good tradeoffs between machine learning accuracy and inference latency. While measuring inference latency of a huge set of candidate architectures during NAS is not feasible, latency prediction for mobile devices is challenging, because of hardware heterogeneity, optimizations applied by machine learning frameworks, and diversity of neural architectures. Motivated by these challenges, we first quantitatively assess the characteristics of neural architectures (specifically, convolutional neural networks for image classification), ML frameworks, and mobile devices that have significant effects on inference latency. Based on this assessment, we propose an operation-wise framework which addresses these challenges by developing operation-wise latency predictors and achieves high accuracy in end-to-end latency predictions, as shown by our comprehensive evaluations on multiple mobile devices using multicore CPUs and GPUs. To illustrate that our approach does not require expensive data collection, we also show that accurate predictions can be achieved on real-world neural architectures using only small amounts of profiling data.
3421.
Man, Yilei; Xie, Linhai; Qiao, Shushan; Zhou, Yumei; Shang, Delong
Differentiable architecture search with multi-dimensional attention for spiking neural networks Journal Article
In: Neurocomputing, vol. 601, pp. 128181, 2024, ISSN: 0925-2312.
@article{MAN2024128181,
title = {Differentiable architecture search with multi-dimensional attention for spiking neural networks},
author = {Yilei Man and Linhai Xie and Shushan Qiao and Yumei Zhou and Delong Shang},
url = {https://www.sciencedirect.com/science/article/pii/S0925231224009524},
doi = {https://doi.org/10.1016/j.neucom.2024.128181},
issn = {0925-2312},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Neurocomputing},
volume = {601},
pages = {128181},
abstract = {Spiking Neural Networks (SNNs) have gained enormous popularity in the field of artificial intelligence due to their low power consumption. However, the majority of SNN methods directly inherit the structure of Artificial Neural Networks (ANN), usually leading to sub-optimal model performance in SNNs. To alleviate this problem, we integrate Neural Architecture Search (NAS) method and propose Multi-Attention Differentiable Architecture Search (MA-DARTS) to directly automate the search for the optimal network structure of SNNs. Initially, we defined a differentiable two-level search space and conducted experiments within micro architecture under a fixed layer. Then, we incorporated a multi-dimensional attention mechanism and implemented the MA-DARTS algorithm in this search space. Comprehensive experiments demonstrate our model achieves state-of-the-art performance on classification compared to other methods under the same parameters with 94.40% accuracy on CIFAR10 dataset and 76.52% accuracy on CIFAR100 dataset. Additionally, we monitored and assessed the number of spikes (NoS) in each cell during the whole experiment. Notably, the number of spikes of the whole model stabilized at approximately 110K in validation and 100k in training on datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Spiking Neural Networks (SNNs) have gained enormous popularity in the field of artificial intelligence due to their low power consumption. However, the majority of SNN methods directly inherit the structure of Artificial Neural Networks (ANN), usually leading to sub-optimal model performance in SNNs. To alleviate this problem, we integrate Neural Architecture Search (NAS) method and propose Multi-Attention Differentiable Architecture Search (MA-DARTS) to directly automate the search for the optimal network structure of SNNs. Initially, we defined a differentiable two-level search space and conducted experiments within micro architecture under a fixed layer. Then, we incorporated a multi-dimensional attention mechanism and implemented the MA-DARTS algorithm in this search space. Comprehensive experiments demonstrate our model achieves state-of-the-art performance on classification compared to other methods under the same parameters with 94.40% accuracy on CIFAR10 dataset and 76.52% accuracy on CIFAR100 dataset. Additionally, we monitored and assessed the number of spikes (NoS) in each cell during the whole experiment. Notably, the number of spikes of the whole model stabilized at approximately 110K in validation and 100k in training on datasets.
3420.
Wang, Li; Yang, Yumeng; Xu, Lili; Ren, Ziyu; Fan, Shurui; Zhang, Yong
A particle swarm optimization-based deep clustering algorithm for power load curve analysis Journal Article
In: Swarm and Evolutionary Computation, vol. 89, pp. 101650, 2024, ISSN: 2210-6502.
@article{WANG2024101650,
title = {A particle swarm optimization-based deep clustering algorithm for power load curve analysis},
author = {Li Wang and Yumeng Yang and Lili Xu and Ziyu Ren and Shurui Fan and Yong Zhang},
url = {https://www.sciencedirect.com/science/article/pii/S2210650224001883},
doi = {https://doi.org/10.1016/j.swevo.2024.101650},
issn = {2210-6502},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Swarm and Evolutionary Computation},
volume = {89},
pages = {101650},
abstract = {To address the inflexibility of the convolutional autoencoder (CAE) in adjusting the network structure and the difficulty of accurately delineating complex class boundaries in power load data, a particle swarm optimization deep clustering method (DC-PSO) is proposed. First, a particle swarm optimization algorithm for automatically searching the optimal network architecture and hyperparameters of CAE (AHPSO) is proposed to obtain better reconstruction performance. Then, an end-to-end deep clustering model based on a reliable sample selection strategy is designed for the deep clustering algorithm to accurately delineate the category boundaries and further improve the clustering effect. The experimental results show that the DC-PSO algorithm exhibits high clustering accuracy and higher performance for the power load profile clustering.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
To address the inflexibility of the convolutional autoencoder (CAE) in adjusting the network structure and the difficulty of accurately delineating complex class boundaries in power load data, a particle swarm optimization deep clustering method (DC-PSO) is proposed. First, a particle swarm optimization algorithm for automatically searching the optimal network architecture and hyperparameters of CAE (AHPSO) is proposed to obtain better reconstruction performance. Then, an end-to-end deep clustering model based on a reliable sample selection strategy is designed for the deep clustering algorithm to accurately delineate the category boundaries and further improve the clustering effect. The experimental results show that the DC-PSO algorithm exhibits high clustering accuracy and higher performance for the power load profile clustering.
3419.
Verma, Prabhat Kumar Sahil; Singh, Jyoti Prakash
MLNAS: Meta-learning based neural architecture search for automated generation of deep neural networks for plant disease detection tasks Journal Article
In: Network: Computation in Neural Systems, vol. 0, no. 0, pp. 1–24, 2024, (PMID: 38994690).
@article{doi:10.1080/0954898X.2024.2374852,
title = {MLNAS: Meta-learning based neural architecture search for automated generation of deep neural networks for plant disease detection tasks},
author = {Prabhat Kumar Sahil Verma and Jyoti Prakash Singh},
url = {https://doi.org/10.1080/0954898X.2024.2374852},
doi = {10.1080/0954898X.2024.2374852},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Network: Computation in Neural Systems},
volume = {0},
number = {0},
pages = {1–24},
publisher = {Taylor & Francis},
note = {PMID: 38994690},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3418.
Annavajjala, Aditya; Khare, Alind; Agrawal, Animesh; Fedorov, Igor; Latapie, Hugo; Lee, Myungjin; Tumanov, Alexey
DepsilonpS: Delayed epsilon-Shrinking for Faster Once-For-All Training Technical Report
2024.
@techreport{annavajjala2024depsilonpsdelayedepsilonshrinkingfaster,
title = {DepsilonpS: Delayed epsilon-Shrinking for Faster Once-For-All Training},
author = {Aditya Annavajjala and Alind Khare and Animesh Agrawal and Igor Fedorov and Hugo Latapie and Myungjin Lee and Alexey Tumanov},
url = {https://arxiv.org/abs/2407.06167},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3417.
Tan, Wenxuan; Roberts, Nicholas; Huang, Tzu-Heng; Zhao, Jitian; Cooper, John; Guo, Samuel; Duan, Chengyu; Sala, Frederic
MoRe Fine-Tuning with 10x Fewer Parameters Proceedings Article
In: Workshop on Efficient Systems for Foundation Models II @ ICML2024, 2024.
@inproceedings{<LineBreak>tan2024more,
title = {MoRe Fine-Tuning with 10x Fewer Parameters},
author = {Wenxuan Tan and Nicholas Roberts and Tzu-Heng Huang and Jitian Zhao and John Cooper and Samuel Guo and Chengyu Duan and Frederic Sala},
url = {https://openreview.net/forum?id=ua9ndTCxme},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Workshop on Efficient Systems for Foundation Models II @ ICML2024},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3416.
Wei, Jiahong; Xue, Bing; Zhang, Mengjie
EZUAS: Evolutionary Zero-shot U-shape Architecture Search for Medical Image Segmentation Proceedings Article
In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 422–430, Association for Computing Machinery, Melbourne, VIC, Australia, 2024, ISBN: 9798400704949.
@inproceedings{10.1145/3638529.3654041,
title = {EZUAS: Evolutionary Zero-shot U-shape Architecture Search for Medical Image Segmentation},
author = {Jiahong Wei and Bing Xue and Mengjie Zhang},
url = {https://doi.org/10.1145/3638529.3654041},
doi = {10.1145/3638529.3654041},
isbn = {9798400704949},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
pages = {422–430},
publisher = {Association for Computing Machinery},
address = {Melbourne, VIC, Australia},
series = {GECCO '24},
abstract = {Recently, deep learning-based methods have become the mainstream for medical image segmentation. Since manually designing deep neural networks (DNNs) is laborious and time-consuming, neural architecture search (NAS) becomes a popular stream for automatically designing DNNs for medical image segmentation. However, existing NAS work for medical image segmentation is still computationally expensive. Given the limited computation power, it is not always applicable to search for a well-performing model from an enlarged search space. In this paper, we propose EZUAS, a novel method of evolutionary zero-shot NAS for medical image segmentation, to address these issues. First, a new search space is designed for the automated design of DNNs. A genetic algorithm (GA) with an aligned crossover operation is then leveraged to search the network architectures under the model complexity constraints to get performant and lightweight models. In addition, a variable-length integer encoding scheme is devised to encode the candidate U-shaped DNNs with different stages. We conduct experiments on two commonly used medical image segmentation datasets to verify the effectiveness of the proposed EZUAS. Compared with the state-of-the-art methods, the proposed method can find a model much faster (about 0.04 GPU day) and achieve the best performance with lower computational complexity.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Recently, deep learning-based methods have become the mainstream for medical image segmentation. Since manually designing deep neural networks (DNNs) is laborious and time-consuming, neural architecture search (NAS) becomes a popular stream for automatically designing DNNs for medical image segmentation. However, existing NAS work for medical image segmentation is still computationally expensive. Given the limited computation power, it is not always applicable to search for a well-performing model from an enlarged search space. In this paper, we propose EZUAS, a novel method of evolutionary zero-shot NAS for medical image segmentation, to address these issues. First, a new search space is designed for the automated design of DNNs. A genetic algorithm (GA) with an aligned crossover operation is then leveraged to search the network architectures under the model complexity constraints to get performant and lightweight models. In addition, a variable-length integer encoding scheme is devised to encode the candidate U-shaped DNNs with different stages. We conduct experiments on two commonly used medical image segmentation datasets to verify the effectiveness of the proposed EZUAS. Compared with the state-of-the-art methods, the proposed method can find a model much faster (about 0.04 GPU day) and achieve the best performance with lower computational complexity.
3415.
Thanh, Tran Hai; Doan, Long; Luong, Ngoc Hoang; Thanh, Binh Huynh Thi
THNAS-GA: A Genetic Algorithm for Training-free Hardware-aware Neural Architecture Search Proceedings Article
In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1128–1136, Association for Computing Machinery, Melbourne, VIC, Australia, 2024, ISBN: 9798400704949.
@inproceedings{10.1145/3638529.3654226,
title = {THNAS-GA: A Genetic Algorithm for Training-free Hardware-aware Neural Architecture Search},
author = {Tran Hai Thanh and Long Doan and Ngoc Hoang Luong and Binh Huynh Thi Thanh},
url = {https://doi.org/10.1145/3638529.3654226},
doi = {10.1145/3638529.3654226},
isbn = {9798400704949},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
pages = {1128–1136},
publisher = {Association for Computing Machinery},
address = {Melbourne, VIC, Australia},
series = {GECCO '24},
abstract = {Neural Architecture Search (NAS) is a promising approach to automate the design of neural network architectures, which can find architectures that perform better than manually designed ones. Hardware-aware NAS is a real-world application of NAS where the architectures found also need to satisfy certain requirements for the deployment of specific devices. Despite the practical importance, hardware-aware NAS still receives a lack of attention from the community. Existing research mostly focuses on the search space with a limited number of architectures, reducing the search process to finding the optimal hyperparameters. In addition, the performance evaluation of found networks is resources-intensive, which can severely hinder reproducibility. In this work, we propose a genetic algorithm approach to the hardware-aware NAS problem, incorporating a latency filtering selection to guarantee the latency validity of candidate solutions. We also introduce an extended search space that can cover various existing architectures from previous research. To speed up the search process, we also present a method to estimate the latency of candidate networks and a training-free performance estimation method to quickly evaluate candidate networks. Our experiments demonstrate that our method achieves competitive performance with state-of-the-art networks while maintaining lower latency with less computation requirements for searching.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) is a promising approach to automate the design of neural network architectures, which can find architectures that perform better than manually designed ones. Hardware-aware NAS is a real-world application of NAS where the architectures found also need to satisfy certain requirements for the deployment of specific devices. Despite the practical importance, hardware-aware NAS still receives a lack of attention from the community. Existing research mostly focuses on the search space with a limited number of architectures, reducing the search process to finding the optimal hyperparameters. In addition, the performance evaluation of found networks is resources-intensive, which can severely hinder reproducibility. In this work, we propose a genetic algorithm approach to the hardware-aware NAS problem, incorporating a latency filtering selection to guarantee the latency validity of candidate solutions. We also introduce an extended search space that can cover various existing architectures from previous research. To speed up the search process, we also present a method to estimate the latency of candidate networks and a training-free performance estimation method to quickly evaluate candidate networks. Our experiments demonstrate that our method achieves competitive performance with state-of-the-art networks while maintaining lower latency with less computation requirements for searching.
3414.
Lv, Zeqiong; Bian, Chao; Qian, Chao; Sun, Yanan
Runtime Analysis of Population-based Evolutionary Neural Architecture Search for a Binary Classification Problem Proceedings Article
In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 358–366, Association for Computing Machinery, Melbourne, VIC, Australia, 2024, ISBN: 9798400704949.
@inproceedings{10.1145/3638529.3654003,
title = {Runtime Analysis of Population-based Evolutionary Neural Architecture Search for a Binary Classification Problem},
author = {Zeqiong Lv and Chao Bian and Chao Qian and Yanan Sun},
url = {https://doi.org/10.1145/3638529.3654003},
doi = {10.1145/3638529.3654003},
isbn = {9798400704949},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
pages = {358–366},
publisher = {Association for Computing Machinery},
address = {Melbourne, VIC, Australia},
series = {GECCO '24},
abstract = {Evolutionary neural architecture search (ENAS) employs evolutionary techniques, e.g., evolutionary algorithm (EA), to design high-performing neural network architectures, and has achieved great success. However, compared to the application, its theoretical analysis is still in its infancy and only touches the ENAS without populations. In this work, we consider the (μ+λ)-ENAS algorithm (based on a general population-based EA with mutation only, i.e., (μ+λ)-EA) to find an optimal neural network architecture capable of solving a binary classification problem Uniform (with problem size n), and obtain the following mathematical runtime results: 1) by applying a local mutation, it can find the optimum in an expected runtime of O(μ + nλ/(1 - e-λ/μ)) and Ω(μ + nλ/(1 - e−-λ)); 2) by applying a global mutation, it can find the optimum in an expected runtime of O(μ + λcλn/(1 - e−-λ/μ)), and Ω(μ + λn ln ln re/ln n) for some constant c > 1. The derived results reveal that the (μ+λ)-ENAS algorithm is always not asymptotically faster than (1+1)-ENAS on the Uniform problem when λ ϵ ω(ln n/(ln ln n)). The concrete theoretical analysis and proof show that increasing the population size has the potential to increase the runtime and thus should be carefully considered in the ENAS algorithm setup.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Evolutionary neural architecture search (ENAS) employs evolutionary techniques, e.g., evolutionary algorithm (EA), to design high-performing neural network architectures, and has achieved great success. However, compared to the application, its theoretical analysis is still in its infancy and only touches the ENAS without populations. In this work, we consider the (μ+λ)-ENAS algorithm (based on a general population-based EA with mutation only, i.e., (μ+λ)-EA) to find an optimal neural network architecture capable of solving a binary classification problem Uniform (with problem size n), and obtain the following mathematical runtime results: 1) by applying a local mutation, it can find the optimum in an expected runtime of O(μ + nλ/(1 - e-λ/μ)) and Ω(μ + nλ/(1 - e−-λ)); 2) by applying a global mutation, it can find the optimum in an expected runtime of O(μ + λcλn/(1 - e−-λ/μ)), and Ω(μ + λn ln ln re/ln n) for some constant c > 1. The derived results reveal that the (μ+λ)-ENAS algorithm is always not asymptotically faster than (1+1)-ENAS on the Uniform problem when λ ϵ ω(ln n/(ln ln n)). The concrete theoretical analysis and proof show that increasing the population size has the potential to increase the runtime and thus should be carefully considered in the ENAS algorithm setup.
3413.
Vo, An; Luong, Ngoc Hoang
Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty Search Proceedings Article
In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1146–1155, Association for Computing Machinery, Melbourne, VIC, Australia, 2024, ISBN: 9798400704949.
@inproceedings{10.1145/3638529.3654064,
title = {Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty Search},
author = {An Vo and Ngoc Hoang Luong},
url = {https://doi.org/10.1145/3638529.3654064},
doi = {10.1145/3638529.3654064},
isbn = {9798400704949},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
pages = {1146–1155},
publisher = {Association for Computing Machinery},
address = {Melbourne, VIC, Australia},
series = {GECCO '24},
abstract = {Neural Architecture Search (NAS) aims to automate the discovery of high-performing deep neural network architectures. Traditional objective-based NAS approaches typically optimize a certain performance metric (e.g., prediction accuracy), overlooking large parts of the architecture search space that potentially contain interesting network configurations. Furthermore, objective-driven population-based metaheuristics in complex search spaces often quickly exhaust population diversity and succumb to premature convergence to local optima. This issue becomes more complicated in NAS when performance objectives do not fully align with the actual performance of the candidate architectures, as is often the case with training-free metrics. While training-free metrics have gained popularity for their rapid performance estimation of candidate architectures without incurring computation-heavy network training, their effective incorporation into NAS remains a challenge. This paper presents the Pareto Dominance-based Novelty Search for multi-objective NAS with Multiple Training-Free metrics (MTF-PDNS). Unlike conventional NAS methods that optimize explicit objectives, MTF-PDNS promotes population diversity by utilizing a novelty score calculated based on multiple training-free performance and complexity metrics, thereby yielding a broader exploration of the search space. Experimental results on standard NAS benchmark suites demonstrate that MTF-PDNS outperforms conventional methods driven by explicit objectives in terms of convergence speed, diversity maintenance, architecture transferability, and computational costs.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) aims to automate the discovery of high-performing deep neural network architectures. Traditional objective-based NAS approaches typically optimize a certain performance metric (e.g., prediction accuracy), overlooking large parts of the architecture search space that potentially contain interesting network configurations. Furthermore, objective-driven population-based metaheuristics in complex search spaces often quickly exhaust population diversity and succumb to premature convergence to local optima. This issue becomes more complicated in NAS when performance objectives do not fully align with the actual performance of the candidate architectures, as is often the case with training-free metrics. While training-free metrics have gained popularity for their rapid performance estimation of candidate architectures without incurring computation-heavy network training, their effective incorporation into NAS remains a challenge. This paper presents the Pareto Dominance-based Novelty Search for multi-objective NAS with Multiple Training-Free metrics (MTF-PDNS). Unlike conventional NAS methods that optimize explicit objectives, MTF-PDNS promotes population diversity by utilizing a novelty score calculated based on multiple training-free performance and complexity metrics, thereby yielding a broader exploration of the search space. Experimental results on standard NAS benchmark suites demonstrate that MTF-PDNS outperforms conventional methods driven by explicit objectives in terms of convergence speed, diversity maintenance, architecture transferability, and computational costs.
3412.
Phan, Quan Minh; Luong, Ngoc Hoang
Efficient Multi-Fidelity Neural Architecture Search with Zero-Cost Proxy-Guided Local Search Proceedings Article
In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 232–240, Association for Computing Machinery, Melbourne, VIC, Australia, 2024, ISBN: 9798400704949.
@inproceedings{10.1145/3638529.3654027,
title = {Efficient Multi-Fidelity Neural Architecture Search with Zero-Cost Proxy-Guided Local Search},
author = {Quan Minh Phan and Ngoc Hoang Luong},
url = {https://doi.org/10.1145/3638529.3654027},
doi = {10.1145/3638529.3654027},
isbn = {9798400704949},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
pages = {232–240},
publisher = {Association for Computing Machinery},
address = {Melbourne, VIC, Australia},
series = {GECCO '24},
abstract = {Using zero-cost (ZC) metrics as proxies for network performance in Neural Architecture Search (NAS) allows search algorithms to thoroughly explore the architecture space due to their low computing costs. Nevertheless, recent studies indicate that relying exclusively on ZC proxies appears to be less effective than using traditional training-based metrics, such as validation accuracy, in seeking high-performance networks. In this study, we investigate the effectiveness of ZC proxies by taking a deeper look into fitness landscapes of ZC proxy-based local searches by utilizing Local Optima Networks (LONs). Our findings exhibit that ZC proxies having high correlation with network performance do not guarantee finding top-performing architectures, and ZC proxies with low correlations could still be better in certain situations. Our results further consolidate the suggestion of favoring training-based metrics over ZC proxies as the search objective. Although we could figure out architectures having the optimal ZC proxy scores, their true performance is often poor. We then propose the Multi-Fidelity Neural Architecture Search (MF-NAS) framework that makes use of the efficiency of ZC proxies and the efficacy of training-based metrics. Experimental results on a wide range of NAS benchmarks demonstrate the superiority of our MF-NAS to state-of-the-art methods under a strict budget.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Using zero-cost (ZC) metrics as proxies for network performance in Neural Architecture Search (NAS) allows search algorithms to thoroughly explore the architecture space due to their low computing costs. Nevertheless, recent studies indicate that relying exclusively on ZC proxies appears to be less effective than using traditional training-based metrics, such as validation accuracy, in seeking high-performance networks. In this study, we investigate the effectiveness of ZC proxies by taking a deeper look into fitness landscapes of ZC proxy-based local searches by utilizing Local Optima Networks (LONs). Our findings exhibit that ZC proxies having high correlation with network performance do not guarantee finding top-performing architectures, and ZC proxies with low correlations could still be better in certain situations. Our results further consolidate the suggestion of favoring training-based metrics over ZC proxies as the search objective. Although we could figure out architectures having the optimal ZC proxy scores, their true performance is often poor. We then propose the Multi-Fidelity Neural Architecture Search (MF-NAS) framework that makes use of the efficiency of ZC proxies and the efficacy of training-based metrics. Experimental results on a wide range of NAS benchmarks demonstrate the superiority of our MF-NAS to state-of-the-art methods under a strict budget.
3411.
Chen, Renqi; Zheng, Xinzhe; Su, Haoyang; Wu, Kehan
HCS-TNAS: Hybrid Constraint-driven Semi-supervised Transformer-NAS for Ultrasound Image Segmentation Technical Report
2024.
@techreport{chen2024hcstnashybridconstraintdrivensemisupervised,
title = {HCS-TNAS: Hybrid Constraint-driven Semi-supervised Transformer-NAS for Ultrasound Image Segmentation},
author = {Renqi Chen and Xinzhe Zheng and Haoyang Su and Kehan Wu},
url = {https://arxiv.org/abs/2407.04203},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3410.
Lin, Chuen-Horng; Chen, Tsung-Yi; Chen, Huan-Yu; Chan, Yung-Kuan
Efficient and lightweight convolutional neural network architecture search methods for object classification Journal Article
In: Pattern Recognition, vol. 156, pp. 110752, 2024, ISSN: 0031-3203.
@article{LIN2024110752,
title = {Efficient and lightweight convolutional neural network architecture search methods for object classification},
author = {Chuen-Horng Lin and Tsung-Yi Chen and Huan-Yu Chen and Yung-Kuan Chan},
url = {https://www.sciencedirect.com/science/article/pii/S003132032400503X},
doi = {https://doi.org/10.1016/j.patcog.2024.110752},
issn = {0031-3203},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Pattern Recognition},
volume = {156},
pages = {110752},
abstract = {Determining the architecture of deep learning models is a complex task. Several automated search techniques have been proposed, but these methods typically require high-performance graphics processing units (GPUs), manual parameter adjustments, and specific training approaches. This study introduces an efficient, lightweight convolutional neural network architecture search approach tailored for object classification. It features an optimized search space design and a novel controller design. This study introduces a refined search space design incorporating optimizations in both spatial and operational aspects. The focus is on the synergistic integration of convolutional units, dimension reduction units, and the stacking of Convolutional Neural Network (CNN) architectures. To enhance the search space, ShuffleNet modules are integrated, reducing the number of parameters and training time. Additionally, BlurPool is implemented in the dimension reduction unit operation to achieve translational invariance, alleviate the gradient vanishing problem, and optimize unit compositions. Moreover, an innovative controller model, Stage LSTM, is proposed based on Long Short-Term Memory (LSTM) to generate lightweight architectural sequences. In conclusion, the refined search space design and the Stage LSTM controller model are synergistically combined to establish an efficient and lightweight architecture search technique termed Stage and Lightweight Network Architecture Search (SLNAS). The experimental results highlight the superior performance of the optimized search space design, primarily when implemented with the Stage LSTM controller model. This approach shows significantly improved accuracy and stability compared to random, traditional LSTM, and Genetic Algorithm (GA) controller models, with statistically significant differences. Notably, the Stage LSTM controller excels in accuracy while producing models with fewer parameters within the expanded architecture search space. The study adopts the Stage LSTM controller model due to its ability to approximate optimal sequence structures, particularly when combined with the optimized search space design, referred to as SLNAS. SLNAS's performance is evaluated through experiments and comparisons with other Neural Architecture Search (NAS) and object classification methods from different researchers. These experiments consider model parameters, hardware resources, model stability, and multiple datasets. The results show that SLNAS achieves a low error rate of 2.86 % on the CIFAR-10 dataset after just 0.2 days of architecture search, matching the performance of manually designed models but using only 2 % of the parameters. SLNAS consistently demonstrates robust performance across various image classification domains, with an approximate parameter count 700,000. To summarize, SLNAS emerges as a highly effective automated network architecture search method tailored for image classification. It streamlines the model design process, making it accessible to researchers without specialized knowledge in deep learning. Optimizing this method unlocks the full potential of deep learning across diverse research areas. Interested parties can publicly access the source code and pre-trained models through the following link: https://github.com/huanyu-chen/LNASG-and-SLNAS-model.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Determining the architecture of deep learning models is a complex task. Several automated search techniques have been proposed, but these methods typically require high-performance graphics processing units (GPUs), manual parameter adjustments, and specific training approaches. This study introduces an efficient, lightweight convolutional neural network architecture search approach tailored for object classification. It features an optimized search space design and a novel controller design. This study introduces a refined search space design incorporating optimizations in both spatial and operational aspects. The focus is on the synergistic integration of convolutional units, dimension reduction units, and the stacking of Convolutional Neural Network (CNN) architectures. To enhance the search space, ShuffleNet modules are integrated, reducing the number of parameters and training time. Additionally, BlurPool is implemented in the dimension reduction unit operation to achieve translational invariance, alleviate the gradient vanishing problem, and optimize unit compositions. Moreover, an innovative controller model, Stage LSTM, is proposed based on Long Short-Term Memory (LSTM) to generate lightweight architectural sequences. In conclusion, the refined search space design and the Stage LSTM controller model are synergistically combined to establish an efficient and lightweight architecture search technique termed Stage and Lightweight Network Architecture Search (SLNAS). The experimental results highlight the superior performance of the optimized search space design, primarily when implemented with the Stage LSTM controller model. This approach shows significantly improved accuracy and stability compared to random, traditional LSTM, and Genetic Algorithm (GA) controller models, with statistically significant differences. Notably, the Stage LSTM controller excels in accuracy while producing models with fewer parameters within the expanded architecture search space. The study adopts the Stage LSTM controller model due to its ability to approximate optimal sequence structures, particularly when combined with the optimized search space design, referred to as SLNAS. SLNAS's performance is evaluated through experiments and comparisons with other Neural Architecture Search (NAS) and object classification methods from different researchers. These experiments consider model parameters, hardware resources, model stability, and multiple datasets. The results show that SLNAS achieves a low error rate of 2.86 % on the CIFAR-10 dataset after just 0.2 days of architecture search, matching the performance of manually designed models but using only 2 % of the parameters. SLNAS consistently demonstrates robust performance across various image classification domains, with an approximate parameter count 700,000. To summarize, SLNAS emerges as a highly effective automated network architecture search method tailored for image classification. It streamlines the model design process, making it accessible to researchers without specialized knowledge in deep learning. Optimizing this method unlocks the full potential of deep learning across diverse research areas. Interested parties can publicly access the source code and pre-trained models through the following link: https://github.com/huanyu-chen/LNASG-and-SLNAS-model.
3409.
Zhang, Xixi; Wang, Yu; Huang, Hao; Lin, Yun; Zhao, Haitao; Gui, Guan
Few-Shot Automatic Modulation Classification Using Architecture Search and Knowledge Transfer in Radar-Communication Coexistence Scenarios Journal Article
In: IEEE Internet of Things Journal, pp. 1-1, 2024.
@article{10584492,
title = {Few-Shot Automatic Modulation Classification Using Architecture Search and Knowledge Transfer in Radar-Communication Coexistence Scenarios},
author = {Xixi Zhang and Yu Wang and Hao Huang and Yun Lin and Haitao Zhao and Guan Gui},
url = {https://ieeexplore.ieee.org/abstract/document/10584492},
doi = {10.1109/JIOT.2024.3423018},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {IEEE Internet of Things Journal},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3408.
Manhas, Varun; Poonam,
Enhancing Smart City Surveillance: Vehicle Number Plate Detection with YOLO NAS Proceedings Article
In: 2024 MIT Art, Design and Technology School of Computing International Conference (MITADTSoCiCon), pp. 1-6, 2024.
@inproceedings{10575200,
title = {Enhancing Smart City Surveillance: Vehicle Number Plate Detection with YOLO NAS},
author = {Varun Manhas and Poonam},
url = {https://ieeexplore.ieee.org/abstract/document/10575200},
doi = {10.1109/MITADTSoCiCon60330.2024.10575200},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 MIT Art, Design and Technology School of Computing International Conference (MITADTSoCiCon)},
pages = {1-6},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3407.
Hoffman, Alexander; Schlichtmann, Ulf; Mueller-Gritschneder, Daniel
MuNAS: TinyML Network Architecture Search Using Goal Attainment and Reinforcement Learning Proceedings Article
In: 2024 13th Mediterranean Conference on Embedded Computing (MECO), pp. 1-8, 2024.
@inproceedings{10577909,
title = {MuNAS: TinyML Network Architecture Search Using Goal Attainment and Reinforcement Learning},
author = {Alexander Hoffman and Ulf Schlichtmann and Daniel Mueller-Gritschneder},
doi = {10.1109/MECO62516.2024.10577909},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 13th Mediterranean Conference on Embedded Computing (MECO)},
pages = {1-8},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3406.
Wu, Hao; Hao, Shuang; Niu, Fenglei; Tu, Jiyuan
Neural network architecture search model for thermal radiation in dense particulate systems Journal Article
In: International Journal of Heat and Fluid Flow, vol. 108, pp. 109498, 2024, ISSN: 0142-727X.
@article{WU2024109498,
title = {Neural network architecture search model for thermal radiation in dense particulate systems},
author = {Hao Wu and Shuang Hao and Fenglei Niu and Jiyuan Tu},
url = {https://www.sciencedirect.com/science/article/pii/S0142727X24002236},
doi = {https://doi.org/10.1016/j.ijheatfluidflow.2024.109498},
issn = {0142-727X},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {International Journal of Heat and Fluid Flow},
volume = {108},
pages = {109498},
abstract = {In the CFD-DEM simulation of the many dense particulate systems, particle-scale thermal radiation is an important heat transfer mode under high temperatures. In this work, neural network architecture search model with different layer connection matrix is applied to the view factor regression of the thermal radiation in dense particulate systems. Deep neural networks with 3346 feasible architectures are evaluated by the HyperBand algorithm to find the local optimal solution. Neural architecture search model trained by the big data of the view factor gives a good prediction of the macroscopic radiative properties and it is in general agreement with the empirical correlations and experimental data. The particle–wall radiation decreases strongly with the distance and the maximum interaction depth is about 2.0 times the sphere diameter. The trained deep neural network model provides an efficient data-driven closure to discuss the thermal radiation of the particle–particle and particle–wall interactions in particle bed.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In the CFD-DEM simulation of the many dense particulate systems, particle-scale thermal radiation is an important heat transfer mode under high temperatures. In this work, neural network architecture search model with different layer connection matrix is applied to the view factor regression of the thermal radiation in dense particulate systems. Deep neural networks with 3346 feasible architectures are evaluated by the HyperBand algorithm to find the local optimal solution. Neural architecture search model trained by the big data of the view factor gives a good prediction of the macroscopic radiative properties and it is in general agreement with the empirical correlations and experimental data. The particle–wall radiation decreases strongly with the distance and the maximum interaction depth is about 2.0 times the sphere diameter. The trained deep neural network model provides an efficient data-driven closure to discuss the thermal radiation of the particle–particle and particle–wall interactions in particle bed.
3405.
Ali, Muhammad Junaid; Moalic, Laurent; Essaid, Mokhtar; Idoumghar, Lhassane
Evolutionary Neural Architecture Search for 2D and 3D Medical Image Classification Proceedings Article
In: Franco, Leonardo; Mulatier, Clélia; Paszynski, Maciej; Krzhizhanovskaya, Valeria V.; Dongarra, Jack J.; Sloot, Peter M. A. (Ed.): Computational Science – ICCS 2024, pp. 131–146, Springer Nature Switzerland, Cham, 2024, ISBN: 978-3-031-63751-3.
@inproceedings{10.1007/978-3-031-63751-3_9,
title = {Evolutionary Neural Architecture Search for 2D and 3D Medical Image Classification},
author = {Muhammad Junaid Ali and Laurent Moalic and Mokhtar Essaid and Lhassane Idoumghar},
editor = {Leonardo Franco and Clélia Mulatier and Maciej Paszynski and Valeria V. Krzhizhanovskaya and Jack J. Dongarra and Peter M. A. Sloot},
url = {https://link.springer.com/chapter/10.1007/978-3-031-63751-3_9},
isbn = {978-3-031-63751-3},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Computational Science – ICCS 2024},
pages = {131–146},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Designing deep learning architectures is a challenging and time-consuming task. To address this problem, Neural Architecture Search (NAS) which automatically searches for a network topology is used. While existing NAS methods mainly focus on image classification tasks, particularly 2D medical images, this study presents an evolutionary NAS approach for 2D and 3D Medical image classification. We defined two different search spaces for 2D and 3D datasets and performed a comparative study of different meta-heuristics used in different NAS studies. Moreover, zero-cost proxies have been used to evaluate the performance of deep neural networks, which helps reduce the searching cost of the overall approach. Furthermore, recognizing the importance of Data Augmentation (DA) in model generalization, we propose a genetic algorithm based automatic DA strategy to find the optimal DA policy. Experiments on MedMNIST benchmark and BreakHIS dataset demonstrate the effectiveness of our approach, showcasing competitive results compared to existing AutoML approaches. The source code of our proposed approach is available at https://github.com/Junaid199f/evo_nas_med_2d_3d.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Designing deep learning architectures is a challenging and time-consuming task. To address this problem, Neural Architecture Search (NAS) which automatically searches for a network topology is used. While existing NAS methods mainly focus on image classification tasks, particularly 2D medical images, this study presents an evolutionary NAS approach for 2D and 3D Medical image classification. We defined two different search spaces for 2D and 3D datasets and performed a comparative study of different meta-heuristics used in different NAS studies. Moreover, zero-cost proxies have been used to evaluate the performance of deep neural networks, which helps reduce the searching cost of the overall approach. Furthermore, recognizing the importance of Data Augmentation (DA) in model generalization, we propose a genetic algorithm based automatic DA strategy to find the optimal DA policy. Experiments on MedMNIST benchmark and BreakHIS dataset demonstrate the effectiveness of our approach, showcasing competitive results compared to existing AutoML approaches. The source code of our proposed approach is available at https://github.com/Junaid199f/evo_nas_med_2d_3d.
3404.
Putra, Rachmad Vidya Wicaksana; Shafique, Muhammad
HASNAS: A Hardware-Aware Spiking Neural Architecture Search Framework for Neuromorphic Compute-in-Memory Systems Technical Report
2024.
@techreport{putra2024hasnashardwareawarespikingneuralb,
title = {HASNAS: A Hardware-Aware Spiking Neural Architecture Search Framework for Neuromorphic Compute-in-Memory Systems},
author = {Rachmad Vidya Wicaksana Putra and Muhammad Shafique},
url = {https://arxiv.org/abs/2407.00641},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3403.
Motetti, Beatrice Alessandra; Risso, Matteo; Burrello, Alessio; Macii, Enrico; Poncino, Massimo; Pagliari, Daniele Jahier
Joint Pruning and Channel-wise Mixed-Precision Quantization for Efficient Deep Neural Networks Technical Report
2024.
@techreport{motetti2024jointpruningchannelwisemixedprecision,
title = {Joint Pruning and Channel-wise Mixed-Precision Quantization for Efficient Deep Neural Networks},
author = {Beatrice Alessandra Motetti and Matteo Risso and Alessio Burrello and Enrico Macii and Massimo Poncino and Daniele Jahier Pagliari},
url = {https://arxiv.org/abs/2407.01054},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3402.
Ahmad, Jamil; Gueaieb, Wail; Saddik, Abdulmotaleb El; Masi, Giulia De; Karray, Fakhri
Yield estimation and health assessment of temperate fruits: A modular framework Journal Article
In: Engineering Applications of Artificial Intelligence, vol. 136, pp. 108871, 2024, ISSN: 0952-1976.
@article{AHMAD2024108871,
title = {Yield estimation and health assessment of temperate fruits: A modular framework},
author = {Jamil Ahmad and Wail Gueaieb and Abdulmotaleb El Saddik and Giulia De Masi and Fakhri Karray},
url = {https://www.sciencedirect.com/science/article/pii/S0952197624010297},
doi = {https://doi.org/10.1016/j.engappai.2024.108871},
issn = {0952-1976},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Engineering Applications of Artificial Intelligence},
volume = {136},
pages = {108871},
abstract = {Yield estimation is crucial for growers and agronomists to optimize crop management practices and facilitate harvest planning. However, traditional manual fruit counting and fruit health assessment activities on large fields are labor-intensive, time-consuming, and prone to errors. Computer vision-based yield estimation methods involving fruit counting and health assessment using unmanned aerial vehicles (UAVs), have gained significant attention in recent years. This study proposes an automated yield estimation and health assessment approach through UAV imaging. Our methodology comprises three main components: (1) a robust fruit detection network based on the “you only look once - neural architecture search” (YOLONAS) model, (2) a fruit health assessment module to detect diseases in individually identified fruits, and (3) a post-processing and regression module for yield quantity and quality estimation. YOLONAS is a computationally efficient and accurate object detection model trained on scale-space augmented datasets. The health assessment module includes separable multiscale convolution layers with an additive attention module. We evaluated our yield estimation approach on three publicly available datasets featuring peach, apple, and citrus trees. Results reveal that YOLONAS, trained with a scale-space augmented dataset, improves detection accuracy by 1.2%. We also used a custom fruit disease dataset to assess the performance of the disease detection model, where we noticed that super-resolution of detected fruits with pre-trained models significantly enhances disease detection by up to 17%, especially in low-resolution fruits. Finally, we demonstrate that the proposed method can serve as a modular framework for yield quantity and quality assessment through UAVs in challenging field conditions.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Yield estimation is crucial for growers and agronomists to optimize crop management practices and facilitate harvest planning. However, traditional manual fruit counting and fruit health assessment activities on large fields are labor-intensive, time-consuming, and prone to errors. Computer vision-based yield estimation methods involving fruit counting and health assessment using unmanned aerial vehicles (UAVs), have gained significant attention in recent years. This study proposes an automated yield estimation and health assessment approach through UAV imaging. Our methodology comprises three main components: (1) a robust fruit detection network based on the “you only look once - neural architecture search” (YOLONAS) model, (2) a fruit health assessment module to detect diseases in individually identified fruits, and (3) a post-processing and regression module for yield quantity and quality estimation. YOLONAS is a computationally efficient and accurate object detection model trained on scale-space augmented datasets. The health assessment module includes separable multiscale convolution layers with an additive attention module. We evaluated our yield estimation approach on three publicly available datasets featuring peach, apple, and citrus trees. Results reveal that YOLONAS, trained with a scale-space augmented dataset, improves detection accuracy by 1.2%. We also used a custom fruit disease dataset to assess the performance of the disease detection model, where we noticed that super-resolution of detected fruits with pre-trained models significantly enhances disease detection by up to 17%, especially in low-resolution fruits. Finally, we demonstrate that the proposed method can serve as a modular framework for yield quantity and quality assessment through UAVs in challenging field conditions.
3401.
Putra, Rachmad Vidya Wicaksana; Shafique, Muhammad
HASNAS: A Hardware-Aware Spiking Neural Architecture Search Framework for Neuromorphic Compute-in-Memory Systems Technical Report
2024.
@techreport{putra2024hasnashardwareawarespikingneural,
title = {HASNAS: A Hardware-Aware Spiking Neural Architecture Search Framework for Neuromorphic Compute-in-Memory Systems},
author = {Rachmad Vidya Wicaksana Putra and Muhammad Shafique},
url = {https://arxiv.org/abs/2407.00641},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3400.
Deligkaris, Kosmas
Particle Swarm Optimization and Random Search for Convolutional Neural Architecture Search Journal Article
In: IEEE Access, vol. 12, pp. 91229-91241, 2024.
@article{10577981,
title = {Particle Swarm Optimization and Random Search for Convolutional Neural Architecture Search},
author = {Kosmas Deligkaris},
url = {https://ieeexplore.ieee.org/document/10577981},
doi = {10.1109/ACCESS.2024.3420870},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {IEEE Access},
volume = {12},
pages = {91229-91241},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3399.
Mehta, Nikhil; Lorraine, Jonathan; Masson, Steve; Arunachalam, Ramanathan; Bhat, Zaid Pervaiz; Lucas, James; Zachariah, Arun George
Improving Hyperparameter Optimization with Checkpointed Model Weights Technical Report
2024.
@techreport{mehta2024improvinghyperparameteroptimizationcheckpointed,
title = {Improving Hyperparameter Optimization with Checkpointed Model Weights},
author = {Nikhil Mehta and Jonathan Lorraine and Steve Masson and Ramanathan Arunachalam and Zaid Pervaiz Bhat and James Lucas and Arun George Zachariah},
url = {https://arxiv.org/abs/2406.18630},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3398.
Ding, Xiangling; Deng, Yingqian; Zhao, Yulin; Zhu, Wenyi
AFTLNet: An efficient adaptive forgery traces learning network for deep image inpainting localization Journal Article
In: Journal of Information Security and Applications, vol. 84, pp. 103825, 2024, ISSN: 2214-2126.
@article{DING2024103825,
title = {AFTLNet: An efficient adaptive forgery traces learning network for deep image inpainting localization},
author = {Xiangling Ding and Yingqian Deng and Yulin Zhao and Wenyi Zhu},
url = {https://www.sciencedirect.com/science/article/pii/S2214212624001285},
doi = {https://doi.org/10.1016/j.jisa.2024.103825},
issn = {2214-2126},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Journal of Information Security and Applications},
volume = {84},
pages = {103825},
abstract = {Deep-learning-based image inpainting repairs a region with visually believable content, leaving behind imperceptible traces. Since deep image inpainting approaches can malevolently remove key objects and erase visible copyright watermarks, the desire for an effective method to distinguish the inpainted regions has become urgent. In this work, we propose an adaptive forgery trace learning network (AFTLN), which consists of two subblocks: the adaptive block and the Densenet block. Specifically, the adaptive block exploits an adaptive difference convolution to maximize the forgery traces by iteratively updating its weights. Meanwhile, the Densenet block improves the feature weights and reduces the impact of noise on the forgery traces. An image-inpainting detector, namely AFTLNet, is designed by integrating AFTLN with neural architecture search, and global and local attention modules, which aims to find potential tampered regions, enhance feature consistency, and reduce intra-class differences, respectively. The experimental results present that our proposed AFTLNet exceeds existing inpainting detection approaches. Finally, an inpainting dataset of 26K image pairs is constructed for future research. The dataset is available at https://pan.baidu.com/s/10SRJeQBNnTHJXvxl8xzHcg with password: 1234.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Deep-learning-based image inpainting repairs a region with visually believable content, leaving behind imperceptible traces. Since deep image inpainting approaches can malevolently remove key objects and erase visible copyright watermarks, the desire for an effective method to distinguish the inpainted regions has become urgent. In this work, we propose an adaptive forgery trace learning network (AFTLN), which consists of two subblocks: the adaptive block and the Densenet block. Specifically, the adaptive block exploits an adaptive difference convolution to maximize the forgery traces by iteratively updating its weights. Meanwhile, the Densenet block improves the feature weights and reduces the impact of noise on the forgery traces. An image-inpainting detector, namely AFTLNet, is designed by integrating AFTLN with neural architecture search, and global and local attention modules, which aims to find potential tampered regions, enhance feature consistency, and reduce intra-class differences, respectively. The experimental results present that our proposed AFTLNet exceeds existing inpainting detection approaches. Finally, an inpainting dataset of 26K image pairs is constructed for future research. The dataset is available at https://pan.baidu.com/s/10SRJeQBNnTHJXvxl8xzHcg with password: 1234.
3397.
Ragusa, Edoardo; Zonzini, Federica; Marchi, Luca De; Zunino, Rodolfo
Compression-Accuracy Co-optimization Through Hardware-aware Neural Architecture Search for Vibration Damage Detection Journal Article
In: IEEE Internet of Things Journal, pp. 1-1, 2024.
@article{10572266,
title = {Compression-Accuracy Co-optimization Through Hardware-aware Neural Architecture Search for Vibration Damage Detection},
author = {Edoardo Ragusa and Federica Zonzini and Luca De Marchi and Rodolfo Zunino},
url = {https://ieeexplore.ieee.org/document/10572266},
doi = {10.1109/JIOT.2024.3419251},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {IEEE Internet of Things Journal},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3396.
Liu, Yang; Liu, Jing; Teng, Yingzhi
Component importance preference-based evolutionary graph neural architecture search Journal Article
In: Information Sciences, vol. 679, pp. 121111, 2024, ISSN: 0020-0255.
@article{LIU2024121111,
title = {Component importance preference-based evolutionary graph neural architecture search},
author = {Yang Liu and Jing Liu and Yingzhi Teng},
url = {https://www.sciencedirect.com/science/article/pii/S0020025524010259},
doi = {https://doi.org/10.1016/j.ins.2024.121111},
issn = {0020-0255},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Information Sciences},
volume = {679},
pages = {121111},
abstract = {Recently, graph neural architecture search (GNAS) has become an increasingly hot research topic as a promising technique for automatically searching graph neural networks (GNNs) with no or little domain expertise. The search space and optimization strategy are the core factors of GNAS. However, the search space of existing GNAS methods is limited, and their optimization strategies treat components indiscriminately. In the current study, a more expressive search space is first designed. Subsequently, it is illustrated that components contribute different importance to data-specific tasks or datasets, and this study assumes component importance as a probability parameter. To this end, a component importance preference-based evolutionary GNAS method (called CIPE) is proposed. CIPE defines component importance and its preference selection and updating method. Subsequently, the designed importance preference-guided multipoint crossover and multistrategy mutation operators are applied to the evolutionary process. Finally, the effectiveness of CIPE is verified for transductive and inductive tasks. The experimental results demonstrate the validity of the component importance assumption and the superiority of CIPE compared with the state-of-the-art handcrafted GNNs and GNAS methods on all eight datasets. The mean accuracy obtained by CIPE on the datasets Cora, CiteSeer, PubMed, Cornell, Texas, Wisconsin, and Chameleon is 83.84%, 73.23%, 80.28%, 78.38%, 86.49%, 82.35%, and 73.59%, respectively. Specifically, the mean accuracy is improved by 2.71% and 5.28% on the datasets Texas and Chameleon, respectively. The mean F1-score obtained by CIPE on the dataset PPI is 99.37%, with an improvement of 0.24%. The code is available at https://github.com/chnyliu/CIPE.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recently, graph neural architecture search (GNAS) has become an increasingly hot research topic as a promising technique for automatically searching graph neural networks (GNNs) with no or little domain expertise. The search space and optimization strategy are the core factors of GNAS. However, the search space of existing GNAS methods is limited, and their optimization strategies treat components indiscriminately. In the current study, a more expressive search space is first designed. Subsequently, it is illustrated that components contribute different importance to data-specific tasks or datasets, and this study assumes component importance as a probability parameter. To this end, a component importance preference-based evolutionary GNAS method (called CIPE) is proposed. CIPE defines component importance and its preference selection and updating method. Subsequently, the designed importance preference-guided multipoint crossover and multistrategy mutation operators are applied to the evolutionary process. Finally, the effectiveness of CIPE is verified for transductive and inductive tasks. The experimental results demonstrate the validity of the component importance assumption and the superiority of CIPE compared with the state-of-the-art handcrafted GNNs and GNAS methods on all eight datasets. The mean accuracy obtained by CIPE on the datasets Cora, CiteSeer, PubMed, Cornell, Texas, Wisconsin, and Chameleon is 83.84%, 73.23%, 80.28%, 78.38%, 86.49%, 82.35%, and 73.59%, respectively. Specifically, the mean accuracy is improved by 2.71% and 5.28% on the datasets Texas and Chameleon, respectively. The mean F1-score obtained by CIPE on the dataset PPI is 99.37%, with an improvement of 0.24%. The code is available at https://github.com/chnyliu/CIPE.
3395.
Ho, Jiacang; Park, Kyongseok; Kang, Dae-Ki
GLNAS: Greedy Layer-wise Network Architecture Search for low cost and fast network generation Journal Article
In: Pattern Recognition, vol. 155, pp. 110730, 2024, ISSN: 0031-3203.
@article{HO2024110730,
title = {GLNAS: Greedy Layer-wise Network Architecture Search for low cost and fast network generation},
author = {Jiacang Ho and Kyongseok Park and Dae-Ki Kang},
url = {https://www.sciencedirect.com/science/article/pii/S0031320324004813},
doi = {https://doi.org/10.1016/j.patcog.2024.110730},
issn = {0031-3203},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Pattern Recognition},
volume = {155},
pages = {110730},
abstract = {The process of applying machine learning algorithms to practical problems can be a challenging and tedious task for non-experts. Previous research has sought to alleviate this burden by introducing automated machine learning techniques, including Network Architecture Search (NAS) and Differentiable Architecture Search (DARTS). However, these methods use a fixed number of layers and predefined skip connections which impose limitations on the generation of an optimal network architecture. In this paper, we propose a novel approach called Greedy Layer-wise Network Architecture Search (GLNAS), which trains network layers one after another and evaluates the network’s performance after each layer is added. GLNAS also assesses the effectiveness of skip connections between layers by testing various outputs of previous layers as an input to the current layer. Our experiment results demonstrate that the network generated by GLNAS requires fewer parameters (i.e., 3.5 millions in both CIFAR-10 and CIFAR-100 datasets) and GPU resources during the searching phase (i.e., 0.17 and 0.24 GPU days in CIFAR-10 and CIFAR-100 datasets respectively) than many existing methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The process of applying machine learning algorithms to practical problems can be a challenging and tedious task for non-experts. Previous research has sought to alleviate this burden by introducing automated machine learning techniques, including Network Architecture Search (NAS) and Differentiable Architecture Search (DARTS). However, these methods use a fixed number of layers and predefined skip connections which impose limitations on the generation of an optimal network architecture. In this paper, we propose a novel approach called Greedy Layer-wise Network Architecture Search (GLNAS), which trains network layers one after another and evaluates the network’s performance after each layer is added. GLNAS also assesses the effectiveness of skip connections between layers by testing various outputs of previous layers as an input to the current layer. Our experiment results demonstrate that the network generated by GLNAS requires fewer parameters (i.e., 3.5 millions in both CIFAR-10 and CIFAR-100 datasets) and GPU resources during the searching phase (i.e., 0.17 and 0.24 GPU days in CIFAR-10 and CIFAR-100 datasets respectively) than many existing methods.
3394.
Jauro, Fatsuma; Gital, Abdulsalam Ya'u; Abdullahi, Usman Ali; Abdulsalami, Aminu Onimisi; Abdullahi, Mohammed; Ibrahim, Adamu Abubakar; Chiroma, Haruna
Modified symbiotic organisms search optimization for automatic construction of convolutional neural network architectures Journal Article
In: Intelligent Systems with Applications, vol. 22, pp. 200349, 2024, ISSN: 2667-3053.
@article{JAURO2024200349,
title = {Modified symbiotic organisms search optimization for automatic construction of convolutional neural network architectures},
author = {Fatsuma Jauro and Abdulsalam Ya'u Gital and Usman Ali Abdullahi and Aminu Onimisi Abdulsalami and Mohammed Abdullahi and Adamu Abubakar Ibrahim and Haruna Chiroma},
url = {https://www.sciencedirect.com/science/article/pii/S2667305324000255},
doi = {https://doi.org/10.1016/j.iswa.2024.200349},
issn = {2667-3053},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Intelligent Systems with Applications},
volume = {22},
pages = {200349},
abstract = {Convolutional Neural Networks (ConvNets) have demonstrated impressive capabilities in image classification; however, the manual creation of these models is a labor-intensive and time-consuming endeavor due to their inherent complexity. This research introduces an innovative approach to Convolutional Neural Network (ConvNet) architecture generation through the utilization of the Symbiotic Organism Search ConvNet (SOS_ConvNet) algorithm. Leveraging the Symbiotic Organism Search optimization technique, SOS_ConvNet evolves ConvNet architectures tailored for diverse image classification tasks. The algorithm's distinctive feature lies in its ability to perform non-numeric computations, rendering it adaptable to intricate deep learning problems. To assess the effectiveness of SOS_ConvNet, experiments were conducted on diverse datasets, including MNIST, Fashion-MNIST, CIFAR-10, and the Breast Cancer dataset. Comparative analysis against existing models showcased the superior performance of SOS_ConvNet in terms of accuracy, error rate, and parameter efficiency. Notably, on the MNIST dataset, SOS_ConvNet achieved an impressive 0.31 % error rate, while on Fashion-MNIST, it demonstrated a competitive 6.7 % error rate, coupled with unparalleled parameter efficiency of 0.24 million parameters. The model excelled on CIFAR-10 and BreakHis datasets, yielding accuracies of 82.78 % and 89.12 %, respectively. Remarkably, the algorithm achieves remarkable accuracy while maintaining moderate model size.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Convolutional Neural Networks (ConvNets) have demonstrated impressive capabilities in image classification; however, the manual creation of these models is a labor-intensive and time-consuming endeavor due to their inherent complexity. This research introduces an innovative approach to Convolutional Neural Network (ConvNet) architecture generation through the utilization of the Symbiotic Organism Search ConvNet (SOS_ConvNet) algorithm. Leveraging the Symbiotic Organism Search optimization technique, SOS_ConvNet evolves ConvNet architectures tailored for diverse image classification tasks. The algorithm's distinctive feature lies in its ability to perform non-numeric computations, rendering it adaptable to intricate deep learning problems. To assess the effectiveness of SOS_ConvNet, experiments were conducted on diverse datasets, including MNIST, Fashion-MNIST, CIFAR-10, and the Breast Cancer dataset. Comparative analysis against existing models showcased the superior performance of SOS_ConvNet in terms of accuracy, error rate, and parameter efficiency. Notably, on the MNIST dataset, SOS_ConvNet achieved an impressive 0.31 % error rate, while on Fashion-MNIST, it demonstrated a competitive 6.7 % error rate, coupled with unparalleled parameter efficiency of 0.24 million parameters. The model excelled on CIFAR-10 and BreakHis datasets, yielding accuracies of 82.78 % and 89.12 %, respectively. Remarkably, the algorithm achieves remarkable accuracy while maintaining moderate model size.
3393.
Zhu, Yudi; Wang, Tao; Li, Zhuoheng; Ni, Wangze; Zhang, Kai; He, Tong; Fu, Michelle; Zeng, Min; Yang, Jianhua; Hu, Nantao; Cai, Wei; Yang, Zhi
Gas identification using electronic nose via gramian-angular-field-based image conversion and convolutional neural networks architecture search Journal Article
In: Sensors and Actuators B: Chemical, vol. 417, pp. 136198, 2024, ISSN: 0925-4005.
@article{ZHU2024136198,
title = {Gas identification using electronic nose via gramian-angular-field-based image conversion and convolutional neural networks architecture search},
author = {Yudi Zhu and Tao Wang and Zhuoheng Li and Wangze Ni and Kai Zhang and Tong He and Michelle Fu and Min Zeng and Jianhua Yang and Nantao Hu and Wei Cai and Zhi Yang},
url = {https://www.sciencedirect.com/science/article/pii/S0925400524009286},
doi = {https://doi.org/10.1016/j.snb.2024.136198},
issn = {0925-4005},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Sensors and Actuators B: Chemical},
volume = {417},
pages = {136198},
abstract = {Recent years have witnessed the splendid performance of deep learning methods used in gas recognition for electronic noses (E-nose). In addition to effective feature extraction, the architecture of the deep neural network plays a vital role. Currently, most applied network structures are hand-crafted by human experts, which is time-consuming and problem-dependent, making it necessary to design the structures of neural networks according to specific demands. In this work, a genetic algorithm with particle swarm optimization (GA-PSO), which possesses promising optimization capabilities, is applied to search for effective deep convolutional neural networks (CNNs) for gas classification based on E-nose technology. A novel image transformation strategy using Gramian angular field and a hybrid cost-saving method is employed in the search process, enabling adaptive and efficient CNN search on gas datasets. With the proposed methods, we can achieve an average classification accuracy of over 90 % on two public gas datasets, while also significantly reducing the model size compared to state-of-the-art CNNs. By using these novel strategies, our approach surpasses random search and basic PSO algorithm in achieving the global optimal solution, higher and more stable accuracy, and faster convergence in pattern recognition using E-nose. Our work suggests that the proposed method can quickly identify excellent CNN structures for E-nose applications.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recent years have witnessed the splendid performance of deep learning methods used in gas recognition for electronic noses (E-nose). In addition to effective feature extraction, the architecture of the deep neural network plays a vital role. Currently, most applied network structures are hand-crafted by human experts, which is time-consuming and problem-dependent, making it necessary to design the structures of neural networks according to specific demands. In this work, a genetic algorithm with particle swarm optimization (GA-PSO), which possesses promising optimization capabilities, is applied to search for effective deep convolutional neural networks (CNNs) for gas classification based on E-nose technology. A novel image transformation strategy using Gramian angular field and a hybrid cost-saving method is employed in the search process, enabling adaptive and efficient CNN search on gas datasets. With the proposed methods, we can achieve an average classification accuracy of over 90 % on two public gas datasets, while also significantly reducing the model size compared to state-of-the-art CNNs. By using these novel strategies, our approach surpasses random search and basic PSO algorithm in achieving the global optimal solution, higher and more stable accuracy, and faster convergence in pattern recognition using E-nose. Our work suggests that the proposed method can quickly identify excellent CNN structures for E-nose applications.
3392.
Yan, Jiaming
Fair Differentiable Neural Network Architecture Search for Long-Tailed Data with Self-Supervised Learning Technical Report
2024.
@techreport{yan2024fairdifferentiableneuralnetwork,
title = {Fair Differentiable Neural Network Architecture Search for Long-Tailed Data with Self-Supervised Learning},
author = {Jiaming Yan},
url = {https://arxiv.org/abs/2406.16949},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3391.
Uddin, A. Hasib; Arif, Abu Shamim Mohammad
Discovering Bengali Consonant-Linked Vowels in Facial Images using Neural Architecture Search Proceedings Article
In: 2024 3rd International Conference on Advancement in Electrical and Electronic Engineering (ICAEEE), pp. 1-6, 2024.
@inproceedings{10561669,
title = {Discovering Bengali Consonant-Linked Vowels in Facial Images using Neural Architecture Search},
author = {A. Hasib Uddin and Abu Shamim Mohammad Arif},
url = {https://ieeexplore.ieee.org/abstract/document/10561669},
doi = {10.1109/ICAEEE62219.2024.10561669},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 3rd International Conference on Advancement in Electrical and Electronic Engineering (ICAEEE)},
pages = {1-6},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3390.
Zhang, Zehuan; Fan, Hongxiang; Chen, Hao Mark; Dudziak, Lukasz; Luk, Wayne
Hardware-Aware Neural Dropout Search for Reliable Uncertainty Prediction on FPGA Technical Report
2024.
@techreport{zhang2024hardwareawareneuraldropoutsearch,
title = {Hardware-Aware Neural Dropout Search for Reliable Uncertainty Prediction on FPGA},
author = {Zehuan Zhang and Hongxiang Fan and Hao Mark Chen and Lukasz Dudziak and Wayne Luk},
url = {https://arxiv.org/abs/2406.16198},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3389.
Yang, Zhengxin; Gao, Wanling; Peng, Luzhou; Huang, Yunyou; Tang, Fei; Zhan, Jianfeng
Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture Technical Report
2024.
@techreport{yang2024youngerdatasetartificialintelligencegenerated,
title = {Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture},
author = {Zhengxin Yang and Wanling Gao and Luzhou Peng and Yunyou Huang and Fei Tang and Jianfeng Zhan},
url = {https://arxiv.org/abs/2406.15132},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}