Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
2452.
Mao, Pengli; Lin, Yan; Zhang, Baochang; Li, Lin
UCB-Based Architecture Search for Remaining Useful Life Prediction Proceedings Article
In: Jia, Yingmin; Zhang, Weicun; Fu, Yongling; Wang, Jiqiang (Ed.): Proceedings of 2023 Chinese Intelligent Systems Conference, pp. 677–686, Springer Nature Singapore, Singapore, 2023, ISBN: 978-981-99-6886-2.
@inproceedings{10.1007/978-981-99-6886-2_58,
title = {UCB-Based Architecture Search for Remaining Useful Life Prediction},
author = {Pengli Mao and Yan Lin and Baochang Zhang and Lin Li},
editor = {Yingmin Jia and Weicun Zhang and Yongling Fu and Jiqiang Wang},
isbn = {978-981-99-6886-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of 2023 Chinese Intelligent Systems Conference},
pages = {677–686},
publisher = {Springer Nature Singapore},
address = {Singapore},
abstract = {Remaining useful life (RUL) is a significant challenge in prognostics and health management. With the development of data science and deep learning, neural network technology has been applied to this problem. In this study, we present a self-optimized mechanism for neural architecture search. Based on upper confidence bound (UCB) algorithm, we construct a reinforcement learning method for search strategy. UCB explores the combinatorial parameter space of a multi-head convolutional neural network with a recurrent neural network to find a suitable architecture. We conducted experiments on the C-MAPSS dataset, and our proposed model achieved better results in measured by RMSE and Scoring function. Compared with other data-based approaches, our method had better performance in terms of both precision and efficiency.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Remaining useful life (RUL) is a significant challenge in prognostics and health management. With the development of data science and deep learning, neural network technology has been applied to this problem. In this study, we present a self-optimized mechanism for neural architecture search. Based on upper confidence bound (UCB) algorithm, we construct a reinforcement learning method for search strategy. UCB explores the combinatorial parameter space of a multi-head convolutional neural network with a recurrent neural network to find a suitable architecture. We conducted experiments on the C-MAPSS dataset, and our proposed model achieved better results in measured by RMSE and Scoring function. Compared with other data-based approaches, our method had better performance in terms of both precision and efficiency.
2451.
Li, Wenhao; Su, Xiu; You, Shan; Wang, Fei; Qian, Chen; Xu, Chang
DiffNAS: Bootstrapping Diffusion Models by Prompting for Better Architectures Booklet
2023.
@booklet{DBLP:journals/corr/abs-2310-04750,
title = {DiffNAS: Bootstrapping Diffusion Models by Prompting for Better Architectures},
author = {Wenhao Li and Xiu Su and Shan You and Fei Wang and Chen Qian and Chang Xu},
url = {https://doi.org/10.48550/arXiv.2310.04750},
doi = {10.48550/ARXIV.2310.04750},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.04750},
month = {01},
keywords = {},
pubstate = {published},
tppubtype = {booklet}
}
2450.
Arhipov, Andrey; Fomin, Ivan
Multi-Objective Fine-Grained Neural Architecture Search Proceedings Article
In: 2023 International Russian Automation Conference (RusAutoCon), pp. 164-169, 2023.
@inproceedings{10272824,
title = {Multi-Objective Fine-Grained Neural Architecture Search},
author = {Andrey Arhipov and Ivan Fomin},
url = {https://ieeexplore.ieee.org/abstract/document/10272824},
doi = {10.1109/RusAutoCon58002.2023.10272824},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 International Russian Automation Conference (RusAutoCon)},
pages = {164-169},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2449.
Lima, Rafael G. De; Freitas, Pedro Garcia; Lucafo, Giovani D.; Fioravanti, Vanessa; Seidel, Ismael; Penatti, Otávio A. B.
Neural Architecture Search for Tiny Detectors of Inter-beat Intervals Proceedings Article
In: 31st European Signal Processing Conference, EUSIPCO 2023, Helsinki, Finland, September 4-8, 2023, pp. 1085–1089, IEEE, 2023.
@inproceedings{DBLP:conf/eusipco/LimaFLFSP23,
title = {Neural Architecture Search for Tiny Detectors of Inter-beat Intervals},
author = {Rafael G. De Lima and Pedro Garcia Freitas and Giovani D. Lucafo and Vanessa Fioravanti and Ismael Seidel and Otávio A. B. Penatti},
url = {https://doi.org/10.23919/EUSIPCO58844.2023.10289754},
doi = {10.23919/EUSIPCO58844.2023.10289754},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {31st European Signal Processing Conference, EUSIPCO 2023, Helsinki,
Finland, September 4-8, 2023},
pages = {1085–1089},
publisher = {IEEE},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2448.
Ghosh, Arjun; Jana, Nanda Dulal; Das, Swagatam; Mallipeddi, Rammohan
Two-Phase Evolutionary Convolutional Neural Network Architecture Search for Medical Image Classification Journal Article
In: IEEE Access, vol. 11, pp. 115280–115305, 2023.
@article{DBLP:journals/access/GhoshJDM23,
title = {Two-Phase Evolutionary Convolutional Neural Network Architecture Search for Medical Image Classification},
author = {Arjun Ghosh and Nanda Dulal Jana and Swagatam Das and Rammohan Mallipeddi},
url = {https://doi.org/10.1109/ACCESS.2023.3323705},
doi = {10.1109/ACCESS.2023.3323705},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Access},
volume = {11},
pages = {115280–115305},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2447.
Kulshrestha, Ankit; Lykov, Danylo; Safro, Ilya; Alexeev, Yuri
QArchSearch: A Scalable Quantum Architecture Search Package Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-07858,
title = {QArchSearch: A Scalable Quantum Architecture Search Package},
author = {Ankit Kulshrestha and Danylo Lykov and Ilya Safro and Yuri Alexeev},
url = {https://doi.org/10.48550/arXiv.2310.07858},
doi = {10.48550/ARXIV.2310.07858},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.07858},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2446.
Sun, Guocheng; Liu, Shitou; Shi, Chaojing; Liu, Xi; Guo, Qianjin
3DCNAS: A universal method for predicting the location of fluorescent organelles in living cells in three-dimensional space Journal Article
In: Experimental Cell Research, vol. 433, no. 2, pp. 113807, 2023, ISSN: 0014-4827.
@article{SUN2023113807,
title = {3DCNAS: A universal method for predicting the location of fluorescent organelles in living cells in three-dimensional space},
author = {Guocheng Sun and Shitou Liu and Chaojing Shi and Xi Liu and Qianjin Guo},
url = {https://www.sciencedirect.com/science/article/pii/S0014482723003555},
doi = {https://doi.org/10.1016/j.yexcr.2023.113807},
issn = {0014-4827},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Experimental Cell Research},
volume = {433},
number = {2},
pages = {113807},
abstract = {Cellular biology research relies on microscopic imaging techniques for studying the complex structures and dynamic processes within cells. Fluorescence microscopy provides high sensitivity and subcellular resolution but has limitations such as photobleaching and sample preparation challenges. Transmission light microscopy offers a label-free alternative but lacks contrast for detailed interpretation. Deep learning methods have shown promise in analyzing cell images and extracting meaningful information. However, accurately learning and simulating diverse subcellular structures remain challenging. In this study, we propose a method named three-dimensional cell neural architecture search (3DCNAS) to predict subcellular structures of fluorescence using unlabeled transmitted light microscope images. By leveraging the automated search capability of differentiable neural architecture search (NAS), our method partially mitigates the issues of overfitting and underfitting caused by the distinct details of various subcellular structures. Furthermore, we apply our method to analyze cell dynamics in genome-edited human induced pluripotent stem cells during mitotic events. This allows us to study the functional roles of organelles and their involvement in cellular processes, contributing to a comprehensive understanding of cell biology and offering insights into disease pathogenesis.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Cellular biology research relies on microscopic imaging techniques for studying the complex structures and dynamic processes within cells. Fluorescence microscopy provides high sensitivity and subcellular resolution but has limitations such as photobleaching and sample preparation challenges. Transmission light microscopy offers a label-free alternative but lacks contrast for detailed interpretation. Deep learning methods have shown promise in analyzing cell images and extracting meaningful information. However, accurately learning and simulating diverse subcellular structures remain challenging. In this study, we propose a method named three-dimensional cell neural architecture search (3DCNAS) to predict subcellular structures of fluorescence using unlabeled transmitted light microscope images. By leveraging the automated search capability of differentiable neural architecture search (NAS), our method partially mitigates the issues of overfitting and underfitting caused by the distinct details of various subcellular structures. Furthermore, we apply our method to analyze cell dynamics in genome-edited human induced pluripotent stem cells during mitotic events. This allows us to study the functional roles of organelles and their involvement in cellular processes, contributing to a comprehensive understanding of cell biology and offering insights into disease pathogenesis.
2445.
Lv, Zeqiong; Qian, Chao; Sun, Yanan
Benchmarking Analysis of Evolutionary Neural Architecture Search Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2023.
@article{10286149,
title = {Benchmarking Analysis of Evolutionary Neural Architecture Search},
author = {Zeqiong Lv and Chao Qian and Yanan Sun},
url = {https://ieeexplore.ieee.org/abstract/document/10286149},
doi = {10.1109/TEVC.2023.3324852},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2444.
Salehin, Imrus; Islam, Md. Shamiul; Saha, Pritom; Noman, S. M.; Tuni, Azra; Hasan, Md. Mehedi; Baten, Md. Abu
AutoML: A Systematic Review on Automated Machine Learning with Neural Architecture Search Journal Article
In: Journal of Information and Intelligence, 2023, ISSN: 2949-7159.
@article{SALEHIN2023,
title = {AutoML: A Systematic Review on Automated Machine Learning with Neural Architecture Search},
author = {Imrus Salehin and Md. Shamiul Islam and Pritom Saha and S. M. Noman and Azra Tuni and Md. Mehedi Hasan and Md. Abu Baten},
url = {https://www.sciencedirect.com/science/article/pii/S2949715923000604},
doi = {https://doi.org/10.1016/j.jiixd.2023.10.002},
issn = {2949-7159},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Journal of Information and Intelligence},
abstract = {AutoML (Automated Machine Learning) is an emerging field that aims to automate the process of building machine learning models. AutoML emerged to increase productivity and efficiency by automating as much as possible the inefficient work that occurs while repeating this process whenever machine learning is applied. In particular, research has been conducted for a long time on technologies that can effectively develop high-quality models by minimizing the intervention of model developers in the process from data preprocessing to algorithm selection and tuning. In this semantic review research, we summarize the data processing requirements for AutoML approaches and provide a detailed explanation. We place greater emphasis on neural architecture search (NAS) as it currently represents a highly popular sub-topic within the field of AutoML. NAS methods use machine learning algorithms to search through a large space of possible architectures and find the one that performs best on a given task. We provide a summary of the performance achieved by representative NAS algorithms on the CIFAR-10, CIFAR-100, ImageNet and well-known benchmark datasets. Additionally, we delve into several noteworthy research directions in NAS methods including one/two-stage NAS, one-shot NAS and joint hyperparameter with architecture optimization. We discussed how the search space size and complexity in NAS can vary depending on the specific problem being addressed. To conclude, we examine several open problems (SOTA problems) within current AutoML methods that assure further investigation in future research.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
AutoML (Automated Machine Learning) is an emerging field that aims to automate the process of building machine learning models. AutoML emerged to increase productivity and efficiency by automating as much as possible the inefficient work that occurs while repeating this process whenever machine learning is applied. In particular, research has been conducted for a long time on technologies that can effectively develop high-quality models by minimizing the intervention of model developers in the process from data preprocessing to algorithm selection and tuning. In this semantic review research, we summarize the data processing requirements for AutoML approaches and provide a detailed explanation. We place greater emphasis on neural architecture search (NAS) as it currently represents a highly popular sub-topic within the field of AutoML. NAS methods use machine learning algorithms to search through a large space of possible architectures and find the one that performs best on a given task. We provide a summary of the performance achieved by representative NAS algorithms on the CIFAR-10, CIFAR-100, ImageNet and well-known benchmark datasets. Additionally, we delve into several noteworthy research directions in NAS methods including one/two-stage NAS, one-shot NAS and joint hyperparameter with architecture optimization. We discussed how the search space size and complexity in NAS can vary depending on the specific problem being addressed. To conclude, we examine several open problems (SOTA problems) within current AutoML methods that assure further investigation in future research.
2443.
Cao, Bin; Zhou, Zirun; Liu, Xin; Hossain, M. Shamim; Lv, Zhihan
Adaptive Multiobjective Evolutionary Neural Architecture Search for GANs Based on Two-Factor Cooperative Mutation Mechanism Proceedings Article
In: Proceedings of the 2023 Workshop on Advanced Multimedia Computing for Smart Manufacturing and Engineering, pp. 71–76, Association for Computing Machinery, Ottawa ON, Canada, 2023, ISBN: 9798400702730.
@inproceedings{10.1145/3606042.3616463,
title = {Adaptive Multiobjective Evolutionary Neural Architecture Search for GANs Based on Two-Factor Cooperative Mutation Mechanism},
author = {Bin Cao and Zirun Zhou and Xin Liu and M. Shamim Hossain and Zhihan Lv},
url = {https://doi.org/10.1145/3606042.3616463},
doi = {10.1145/3606042.3616463},
isbn = {9798400702730},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 2023 Workshop on Advanced Multimedia Computing for Smart Manufacturing and Engineering},
pages = {71–76},
publisher = {Association for Computing Machinery},
address = {Ottawa ON, Canada},
series = {AMC-SME '23},
abstract = {The automated design of generative adversarial networks (GAN) is currently being solved well by neural architecture search (NAS), although there are still some issues. One problem is the vast majority of NAS for GANs methods are only based on a single evaluation metric or a linear superposition of multiple evaluation metrics. Another problem is that the conventional evolutionary neural architecture search (ENAS) is unable to adjust its mutation probabilities in accordance with the NAS process, making it simple to settle into a local optimum. To address these issues, we firstly design a two-factor cooperative mutation mechanism that can control the mutation probability based on the current iteration rounds of the population, population fitness and other information. Secondly, we divide the evolutionary process into three stages based on the properties of NAS, so that the different stages can adaptively adjust the mutation probability according to the population state and the expected development goals. Finally, we incorporate multiple optimization objectives from GANs based on image generation tasks into ENAS. And we construct an adaptive multiobjective ENAS based on a two-factor cooperative mutation mechanism. We test and ablate our algorithm on the STL-10 and CIFAR-10 datasets, and the experimental results show that our method outperforms the majority of traditional NAS-GANs.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The automated design of generative adversarial networks (GAN) is currently being solved well by neural architecture search (NAS), although there are still some issues. One problem is the vast majority of NAS for GANs methods are only based on a single evaluation metric or a linear superposition of multiple evaluation metrics. Another problem is that the conventional evolutionary neural architecture search (ENAS) is unable to adjust its mutation probabilities in accordance with the NAS process, making it simple to settle into a local optimum. To address these issues, we firstly design a two-factor cooperative mutation mechanism that can control the mutation probability based on the current iteration rounds of the population, population fitness and other information. Secondly, we divide the evolutionary process into three stages based on the properties of NAS, so that the different stages can adaptively adjust the mutation probability according to the population state and the expected development goals. Finally, we incorporate multiple optimization objectives from GANs based on image generation tasks into ENAS. And we construct an adaptive multiobjective ENAS based on a two-factor cooperative mutation mechanism. We test and ablate our algorithm on the STL-10 and CIFAR-10 datasets, and the experimental results show that our method outperforms the majority of traditional NAS-GANs.
2442.
Das, Richeek; Dooley, Samuel
Fairer and More Accurate Tabular Models Through NAS Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-12145,
title = {Fairer and More Accurate Tabular Models Through NAS},
author = {Richeek Das and Samuel Dooley},
url = {https://doi.org/10.48550/arXiv.2310.12145},
doi = {10.48550/ARXIV.2310.12145},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.12145},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2441.
Yao, Peng; Liao, Chao; Jia, Jiyuan; Tan, Jianchao; Chen, Bin; Song, Chengru; Zhang, Di
ASP: Automatic Selection of Proxy dataset for efficient AutoML Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-11478,
title = {ASP: Automatic Selection of Proxy dataset for efficient AutoML},
author = {Peng Yao and Chao Liao and Jiyuan Jia and Jianchao Tan and Bin Chen and Chengru Song and Di Zhang},
url = {https://doi.org/10.48550/arXiv.2310.11478},
doi = {10.48550/ARXIV.2310.11478},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.11478},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2440.
Yu, Caiyang; Wang, Yixi; Tang, Chenwei; Feng, Wentao; Lv, Jiancheng
EU-Net: Automatic U-Net neural architecture search with differential evolutionary algorithm for medical image segmentation Journal Article
In: Computers in Biology and Medicine, vol. 167, pp. 107579, 2023, ISSN: 0010-4825.
@article{YU2023107579,
title = {EU-Net: Automatic U-Net neural architecture search with differential evolutionary algorithm for medical image segmentation},
author = {Caiyang Yu and Yixi Wang and Chenwei Tang and Wentao Feng and Jiancheng Lv},
url = {https://www.sciencedirect.com/science/article/pii/S0010482523010442},
doi = {https://doi.org/10.1016/j.compbiomed.2023.107579},
issn = {0010-4825},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Computers in Biology and Medicine},
volume = {167},
pages = {107579},
abstract = {Medical images are crucial in clinical practice, providing essential information for patient assessment and treatment planning. However, manual extraction of information from images is both time-consuming and prone to errors. The emergence of U-Net addresses this challenge by automating the segmentation of anatomical structures and pathological lesions in medical images, thereby significantly enhancing the accuracy of image interpretation and diagnosis. However, the performance of U-Net largely depends on its encoder–decoder structure, which requires researchers with knowledge of neural network architecture design and an in-depth understanding of medical images. In this paper, we propose an automatic U-Net Neural Architecture Search (NAS) algorithm using the differential evolutionary (DE) algorithm, named EU-Net, to segment critical information in medical images to assist physicians in diagnosis. Specifically, by presenting the variable-length strategy, the proposed EU-Net algorithm can sufficiently and automatically search for the neural network architecture without expertise. Moreover, the utilization of crossover, mutation, and selection strategies of DE takes account of the trade-off between exploration and exploitation in the search space. Finally, in the encoding and decoding phases of the proposed algorithm, different block-based and layer-based structures are introduced for architectural optimization. The proposed EU-Net algorithm is validated on two widely used medical datasets, i.e., CHAOS and BUSI, for image segmentation tasks. Extensive experimental results show that the proposed EU-Net algorithm outperforms the chosen peer competitors in both two datasets. In particular, compared to the original U-Net, our proposed method improves the metric mIou by at least 6%.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Medical images are crucial in clinical practice, providing essential information for patient assessment and treatment planning. However, manual extraction of information from images is both time-consuming and prone to errors. The emergence of U-Net addresses this challenge by automating the segmentation of anatomical structures and pathological lesions in medical images, thereby significantly enhancing the accuracy of image interpretation and diagnosis. However, the performance of U-Net largely depends on its encoder–decoder structure, which requires researchers with knowledge of neural network architecture design and an in-depth understanding of medical images. In this paper, we propose an automatic U-Net Neural Architecture Search (NAS) algorithm using the differential evolutionary (DE) algorithm, named EU-Net, to segment critical information in medical images to assist physicians in diagnosis. Specifically, by presenting the variable-length strategy, the proposed EU-Net algorithm can sufficiently and automatically search for the neural network architecture without expertise. Moreover, the utilization of crossover, mutation, and selection strategies of DE takes account of the trade-off between exploration and exploitation in the search space. Finally, in the encoding and decoding phases of the proposed algorithm, different block-based and layer-based structures are introduced for architectural optimization. The proposed EU-Net algorithm is validated on two widely used medical datasets, i.e., CHAOS and BUSI, for image segmentation tasks. Extensive experimental results show that the proposed EU-Net algorithm outperforms the chosen peer competitors in both two datasets. In particular, compared to the original U-Net, our proposed method improves the metric mIou by at least 6%.
2439.
Zou, Juan; Wu, Shenghong; Xia, Yizhang; Jiang, Weiwei; Wu, Zeping; Zheng, Jinhua
TS-ENAS: Two-Stage Evolution for Cell-based Network Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-09525,
title = {TS-ENAS: Two-Stage Evolution for Cell-based Network Architecture Search},
author = {Juan Zou and Shenghong Wu and Yizhang Xia and Weiwei Jiang and Zeping Wu and Jinhua Zheng},
url = {https://doi.org/10.48550/arXiv.2310.09525},
doi = {10.48550/ARXIV.2310.09525},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.09525},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2438.
Liu, Xiaoyun; Saxena, Divya; Cao, Jiannong; Zhao, Yuqing; Ruan, Penghui
MGAS: Multi-Granularity Architecture Search for Effective and Efficient Neural Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-15074,
title = {MGAS: Multi-Granularity Architecture Search for Effective and Efficient Neural Networks},
author = {Xiaoyun Liu and Divya Saxena and Jiannong Cao and Yuqing Zhao and Penghui Ruan},
url = {https://doi.org/10.48550/arXiv.2310.15074},
doi = {10.48550/ARXIV.2310.15074},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.15074},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2437.
Jawahar, Ganesh; Abdul-Mageed, Muhammad; Lakshmanan, Laks V. S.; Ding, Dujian
LLM Performance Predictors are good initializers for Architecture Search Technical Report
2023.
@techreport{jawahar2023llm,
title = {LLM Performance Predictors are good initializers for Architecture Search},
author = {Ganesh Jawahar and Muhammad Abdul-Mageed and Laks V. S. Lakshmanan and Dujian Ding},
url = {https://arxiv.org/abs/2310.16712},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2436.
Chikwendu, Ijeoma Amuche; Zhang, Xiaoling; Agyemang, Isaac Osei; Mensah, Isaac Adjei; Ukwuoma, Chiagoziem Chima; Ejiyi, Chukwuebuka Joseph
A Comprehensive Survey on Deep Graph Representation Learning Methods Journal Article
In: J. Artif. Intell. Res., vol. 78, pp. 287–356, 2023.
@article{DBLP:journals/jair/ChikwenduZAMUE23,
title = {A Comprehensive Survey on Deep Graph Representation Learning Methods},
author = {Ijeoma Amuche Chikwendu and Xiaoling Zhang and Isaac Osei Agyemang and Isaac Adjei Mensah and Chiagoziem Chima Ukwuoma and Chukwuebuka Joseph Ejiyi},
url = {https://doi.org/10.1613/jair.1.14768},
doi = {10.1613/JAIR.1.14768},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {J. Artif. Intell. Res.},
volume = {78},
pages = {287–356},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2435.
Liang, Yinan; Wang, Ziwei; Xu, Xiuwei; Tang, Yansong; Zhou, Jie; Lu, Jiwen
MCUFormer: Deploying Vision Tranformers on Microcontrollers with Limited Memory Technical Report
2023.
@techreport{liang2023mcuformer,
title = {MCUFormer: Deploying Vision Tranformers on Microcontrollers with Limited Memory},
author = {Yinan Liang and Ziwei Wang and Xiuwei Xu and Yansong Tang and Jie Zhou and Jiwen Lu},
url = {https://arxiv.org/abs/2310.16898},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2434.
Liu, Xiao; Yao, Chong; Chen, Hongyi; Xiang, Rui; Wu, Hao; Du, Peng; Yu, Zekuan; Liu, Weifan; Liu, Jie; Geng, Daoying
BTSC-TNAS: A neural architecture search-based transformer for brain tumor segmentation and classification Journal Article
In: Computerized Medical Imaging and Graphics, vol. 110, pp. 102307, 2023, ISSN: 0895-6111.
@article{LIU2023102307,
title = {BTSC-TNAS: A neural architecture search-based transformer for brain tumor segmentation and classification},
author = {Xiao Liu and Chong Yao and Hongyi Chen and Rui Xiang and Hao Wu and Peng Du and Zekuan Yu and Weifan Liu and Jie Liu and Daoying Geng},
url = {https://www.sciencedirect.com/science/article/pii/S0895611123001258},
doi = {https://doi.org/10.1016/j.compmedimag.2023.102307},
issn = {0895-6111},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Computerized Medical Imaging and Graphics},
volume = {110},
pages = {102307},
abstract = {Glioblastoma (GBM), isolated brain metastasis (SBM), and primary central nervous system lymphoma (PCNSL) possess a high level of similarity in histomorphology and clinical manifestations on multimodal MRI. Such similarities have led to challenges in the clinical diagnosis of these three malignant tumors. However, many existing models solely focus on either the task of segmentation or classification, which limits the application of computer-aided diagnosis in clinical diagnosis and treatment. To solve this problem, we propose a multi-task learning transformer with neural architecture search (NAS) for brain tumor segmentation and classification (BTSC-TNAS). In the segmentation stage, we use a nested transformer U-shape network (NTU-NAS) with NAS to directly predict brain tumor masks from multi-modal MRI images. In the tumor classification stage, we use the multiscale features obtained from the encoder of NTU-NAS as the input features of the classification network (MSC-NET), which are integrated and corrected by the classification feature correction enhancement (CFCE) block to improve the accuracy of classification. The proposed BTSC-TNAS achieves an average Dice coefficient of 80.86% and 87.12% for the segmentation of tumor region and the maximum abnormal region in clinical data respectively. The model achieves a classification accuracy of 0.941. The experiments performed on the BraTS 2019 dataset show that the proposed BTSC-TNAS has excellent generalizability and can provide support for some challenging tasks in the diagnosis and treatment of brain tumors.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Glioblastoma (GBM), isolated brain metastasis (SBM), and primary central nervous system lymphoma (PCNSL) possess a high level of similarity in histomorphology and clinical manifestations on multimodal MRI. Such similarities have led to challenges in the clinical diagnosis of these three malignant tumors. However, many existing models solely focus on either the task of segmentation or classification, which limits the application of computer-aided diagnosis in clinical diagnosis and treatment. To solve this problem, we propose a multi-task learning transformer with neural architecture search (NAS) for brain tumor segmentation and classification (BTSC-TNAS). In the segmentation stage, we use a nested transformer U-shape network (NTU-NAS) with NAS to directly predict brain tumor masks from multi-modal MRI images. In the tumor classification stage, we use the multiscale features obtained from the encoder of NTU-NAS as the input features of the classification network (MSC-NET), which are integrated and corrected by the classification feature correction enhancement (CFCE) block to improve the accuracy of classification. The proposed BTSC-TNAS achieves an average Dice coefficient of 80.86% and 87.12% for the segmentation of tumor region and the maximum abnormal region in clinical data respectively. The model achieves a classification accuracy of 0.941. The experiments performed on the BraTS 2019 dataset show that the proposed BTSC-TNAS has excellent generalizability and can provide support for some challenging tasks in the diagnosis and treatment of brain tumors.
2433.
Zhou, Yuanding; Li, Xinran; Fang, Yaodong; Qin, Chuan
When Perceptual Authentication Hashing Meets Neural Architecture Search Proceedings Article
In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 8975–8983, Association for Computing Machinery, Ottawa ON, Canada, 2023, ISBN: 9798400701085.
@inproceedings{10.1145/3581783.3612457,
title = {When Perceptual Authentication Hashing Meets Neural Architecture Search},
author = {Yuanding Zhou and Xinran Li and Yaodong Fang and Chuan Qin},
url = {https://doi.org/10.1145/3581783.3612457},
doi = {10.1145/3581783.3612457},
isbn = {9798400701085},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 31st ACM International Conference on Multimedia},
pages = {8975–8983},
publisher = {Association for Computing Machinery},
address = {Ottawa ON, Canada},
series = {MM '23},
abstract = {In recent years, many perceptual authentication hashing schemes have been proposed, especially for image content authentication. However, most of the schemes directly use the dataset of image processing during model training and evaluation, which is actually unreasonable due to the task difference. In this paper, we first propose a specialized dataset for perceptual authentication hashing of images (PAHI), and the image content-preserving manipulations used in this dataset are richer and more in line with realistic scenarios. Then, in order to achieve satisfactory perceptual robustness and discrimination capability of PAHI, we exploit the continuous neural architecture search (NAS) on the channel number and stack depth of the ConvNeXt architecture, and obtain two PAHI architectures i.e., NASRes and NASCoNt. The former has better overall performance, while the latter is better for some special manipulations such as image cropping and background overlap. Experimental results demonstrate that our architectures both can achieve competitive results compared with SOTA schemes, and the AUC areas are increased by 1.6 (NASCoNt) and 1.7 (NASRes), respectively.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In recent years, many perceptual authentication hashing schemes have been proposed, especially for image content authentication. However, most of the schemes directly use the dataset of image processing during model training and evaluation, which is actually unreasonable due to the task difference. In this paper, we first propose a specialized dataset for perceptual authentication hashing of images (PAHI), and the image content-preserving manipulations used in this dataset are richer and more in line with realistic scenarios. Then, in order to achieve satisfactory perceptual robustness and discrimination capability of PAHI, we exploit the continuous neural architecture search (NAS) on the channel number and stack depth of the ConvNeXt architecture, and obtain two PAHI architectures i.e., NASRes and NASCoNt. The former has better overall performance, while the latter is better for some special manipulations such as image cropping and background overlap. Experimental results demonstrate that our architectures both can achieve competitive results compared with SOTA schemes, and the AUC areas are increased by 1.6 (NASCoNt) and 1.7 (NASRes), respectively.
2432.
Wang, Wenna; Zhuo, Tao; Zhang, Xiuwei; Sun, Mingjun; Yin, Hanlin; Xing, Yinghui; Zhang, Yanning
Automatic Network Architecture Search for RGB-D Semantic Segmentation Proceedings Article
In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 3777–3786, Association for Computing Machinery, Ottawa ON, Canada, 2023, ISBN: 9798400701085.
@inproceedings{10.1145/3581783.3612288,
title = {Automatic Network Architecture Search for RGB-D Semantic Segmentation},
author = {Wenna Wang and Tao Zhuo and Xiuwei Zhang and Mingjun Sun and Hanlin Yin and Yinghui Xing and Yanning Zhang},
url = {https://doi.org/10.1145/3581783.3612288},
doi = {10.1145/3581783.3612288},
isbn = {9798400701085},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 31st ACM International Conference on Multimedia},
pages = {3777–3786},
publisher = {Association for Computing Machinery},
address = {Ottawa ON, Canada},
series = {MM '23},
abstract = {Recent RGB-D semantic segmentation networks are usually manually designed. However, due to limited human efforts and time costs, their performance might be inferior for complex scenarios. To address this issue, we propose the first Neural Architecture Search (NAS) method that designs the network automatically. Specifically, the target network consists of an encoder and a decoder. The encoder is designed with two independent branches, where each branch specializes in extracting features from RGB and depth images, respectively. The decoder fuses the features and generates the final segmentation result. Besides, for automatic network design, we design a grid-like network-level search space combined with a hierarchical cell-level search space. By further developing an effective gradient-based search strategy, the network structure with hierarchical cell architectures is discovered. Extensive results on two datasets show that the proposed method outperforms the state-of-the-art approaches, which achieves a mIoU score of 55.1% on the NYU-Depth v2 dataset and 50.3% on the SUN-RGBD dataset.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Recent RGB-D semantic segmentation networks are usually manually designed. However, due to limited human efforts and time costs, their performance might be inferior for complex scenarios. To address this issue, we propose the first Neural Architecture Search (NAS) method that designs the network automatically. Specifically, the target network consists of an encoder and a decoder. The encoder is designed with two independent branches, where each branch specializes in extracting features from RGB and depth images, respectively. The decoder fuses the features and generates the final segmentation result. Besides, for automatic network design, we design a grid-like network-level search space combined with a hierarchical cell-level search space. By further developing an effective gradient-based search strategy, the network structure with hierarchical cell architectures is discovered. Extensive results on two datasets show that the proposed method outperforms the state-of-the-art approaches, which achieves a mIoU score of 55.1% on the NYU-Depth v2 dataset and 50.3% on the SUN-RGBD dataset.
2431.
Zhang, Jiayi; Li, Weixin
Multi-Modal and Multi-Scale Temporal Fusion Architecture Search for Audio-Visual Video Parsing Proceedings Article
In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 3328–3336, Association for Computing Machinery, Ottawa ON, Canada, 2023, ISBN: 9798400701085.
@inproceedings{10.1145/3581783.3611947,
title = {Multi-Modal and Multi-Scale Temporal Fusion Architecture Search for Audio-Visual Video Parsing},
author = {Jiayi Zhang and Weixin Li},
url = {https://doi.org/10.1145/3581783.3611947},
doi = {10.1145/3581783.3611947},
isbn = {9798400701085},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 31st ACM International Conference on Multimedia},
pages = {3328–3336},
publisher = {Association for Computing Machinery},
address = {Ottawa ON, Canada},
series = {MM '23},
abstract = {The weakly supervised audio-visual video parsing (AVVP) task aims to parse a video into a set of modality-wise events (i.e., audible, visible, or both), recognize categories of these events, and localize their temporal boundaries. Given the prevalence of audio-visual synchronous and asynchronous contents in multi-modal videos, it is crucial to capture and integrate the contextual events occurring at different moments and temporal scales. Although some researchers have made preliminary attempts at modeling event semantics with various temporal lengths, they mostly only perform a late fusion of multi-scale features across modalities. A comprehensive cross-modal and multi-scale temporal fusion strategy remains largely unexplored in the literature. To address this gap, we propose a novel framework named Audio-Visual Fusion Architecture Search (AVFAS) that can automatically find the optimal multi-scale temporal fusion strategy within and between modalities. Our framework generates a set of audio and visual features with distinct temporal scales and employs three modality-wise modules to search multi-scale feature selection and fusion strategies, jointly modeling modality-specific discriminative information. Furthermore, to enhance the alignment of audio-visual asynchrony, we introduce a Position- and Length-Adaptive Temporal Attention (PLATA) mechanism for cross-modal feature fusion. Extensive quantitative and qualitative experimental results demonstrate the effectiveness and efficiency of our framework.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The weakly supervised audio-visual video parsing (AVVP) task aims to parse a video into a set of modality-wise events (i.e., audible, visible, or both), recognize categories of these events, and localize their temporal boundaries. Given the prevalence of audio-visual synchronous and asynchronous contents in multi-modal videos, it is crucial to capture and integrate the contextual events occurring at different moments and temporal scales. Although some researchers have made preliminary attempts at modeling event semantics with various temporal lengths, they mostly only perform a late fusion of multi-scale features across modalities. A comprehensive cross-modal and multi-scale temporal fusion strategy remains largely unexplored in the literature. To address this gap, we propose a novel framework named Audio-Visual Fusion Architecture Search (AVFAS) that can automatically find the optimal multi-scale temporal fusion strategy within and between modalities. Our framework generates a set of audio and visual features with distinct temporal scales and employs three modality-wise modules to search multi-scale feature selection and fusion strategies, jointly modeling modality-specific discriminative information. Furthermore, to enhance the alignment of audio-visual asynchrony, we introduce a Position- and Length-Adaptive Temporal Attention (PLATA) mechanism for cross-modal feature fusion. Extensive quantitative and qualitative experimental results demonstrate the effectiveness and efficiency of our framework.
2430.
Miranda, Thiago Z.; Sardinha, Diorge B.; Neri, Ferrante; Basgalupp, Márcio P.; Cerri, Ricardo
A Grammar-based multi-objective neuroevolutionary algorithm to generate fully convolutional networks with novel topologies Journal Article
In: Applied Soft Computing, vol. 149, pp. 110967, 2023, ISSN: 1568-4946.
@article{MIRANDA2023110967,
title = {A Grammar-based multi-objective neuroevolutionary algorithm to generate fully convolutional networks with novel topologies},
author = {Thiago Z. Miranda and Diorge B. Sardinha and Ferrante Neri and Márcio P. Basgalupp and Ricardo Cerri},
url = {https://www.sciencedirect.com/science/article/pii/S1568494623009857},
doi = {https://doi.org/10.1016/j.asoc.2023.110967},
issn = {1568-4946},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Applied Soft Computing},
volume = {149},
pages = {110967},
abstract = {The design of complex and deep neural networks is often performed by identifying and combining building blocks and progressively selecting the most promising combination. Neuroevolution automates this process by employing evolutionary algorithms to guide the search. Within this field, grammar-based evolutionary algorithms have been demonstrated to be powerful tools to describe and thus encode complex neural architectures effectively. Following this trend, the present work proposes a novel grammar-based multi-objective neuroevolutionary for generating Fully Convolutional Networks. The proposed method, named Multi-Objective gRammatical Evolution for FUlly convolutional Networks (MOREFUN), includes a new efficient way to encode skip connections, facilitating the description of complex search spaces and the injection of domain knowledge in the search procedure, generation of fully convolutional networks, upsampling of lower-resolution inputs in multi-input layers, usage of multi-objective fitness, and inclusion of data augmentation and optimiser settings to the grammar. Our best networks outperformed previous grammar evolution algorithms, achieving 90.5% accuracy on CIFAR-10 without using transfer learning, ensembles, or test-time data augmentation. Our best models had 13.39±5.25 trainable parameters and the evolutionary process required 90 min per generation.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The design of complex and deep neural networks is often performed by identifying and combining building blocks and progressively selecting the most promising combination. Neuroevolution automates this process by employing evolutionary algorithms to guide the search. Within this field, grammar-based evolutionary algorithms have been demonstrated to be powerful tools to describe and thus encode complex neural architectures effectively. Following this trend, the present work proposes a novel grammar-based multi-objective neuroevolutionary for generating Fully Convolutional Networks. The proposed method, named Multi-Objective gRammatical Evolution for FUlly convolutional Networks (MOREFUN), includes a new efficient way to encode skip connections, facilitating the description of complex search spaces and the injection of domain knowledge in the search procedure, generation of fully convolutional networks, upsampling of lower-resolution inputs in multi-input layers, usage of multi-objective fitness, and inclusion of data augmentation and optimiser settings to the grammar. Our best networks outperformed previous grammar evolution algorithms, achieving 90.5% accuracy on CIFAR-10 without using transfer learning, ensembles, or test-time data augmentation. Our best models had 13.39±5.25 trainable parameters and the evolutionary process required 90 min per generation.
2429.
King, Tobias; Zhou, Yexu; Röddiger, Tobias; Beigl, Michael
MicroNAS: Memory and Latency Constrained Hardware-Aware Neural Architecture Search for Time Series Classification on Microcontrollers Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-18384,
title = {MicroNAS: Memory and Latency Constrained Hardware-Aware Neural Architecture Search for Time Series Classification on Microcontrollers},
author = {Tobias King and Yexu Zhou and Tobias Röddiger and Michael Beigl},
url = {https://doi.org/10.48550/arXiv.2310.18384},
doi = {10.48550/ARXIV.2310.18384},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.18384},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2428.
Garcia-Garcia, Cosijopii; Morales-Reyes, Alicia; Escalante, Hugo Jair
Continuous Cartesian Genetic Programming based representation for Multi-Objective Neural Architecture Search Technical Report
2023.
@techreport{garciagarcia2023continuous,
title = {Continuous Cartesian Genetic Programming based representation for Multi-Objective Neural Architecture Search},
author = {Cosijopii Garcia-Garcia and Alicia Morales-Reyes and Hugo Jair Escalante},
url = {https://arxiv.org/abs/2306.02648},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2427.
Cao, Yufan; Zhang, Tunhou; Wen, Wei; Yan, Feng; Li, Hai; Chen, Yiran
Farthest Greedy Path Sampling for Two-shot Recommender Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-20705,
title = {Farthest Greedy Path Sampling for Two-shot Recommender Search},
author = {Yufan Cao and Tunhou Zhang and Wei Wen and Feng Yan and Hai Li and Yiran Chen},
url = {https://doi.org/10.48550/arXiv.2310.20705},
doi = {10.48550/ARXIV.2310.20705},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.20705},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2426.
Ding, Li; Zoghi, Masrour; Tennenholtz, Guy; Karimzadehgan, Maryam
Ever Evolving Evaluator (EV3): Towards Flexible and Reliable Meta-Optimization for Knowledge Distillation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2310-18893,
title = {Ever Evolving Evaluator (EV3): Towards Flexible and Reliable Meta-Optimization for Knowledge Distillation},
author = {Li Ding and Masrour Zoghi and Guy Tennenholtz and Maryam Karimzadehgan},
url = {https://doi.org/10.48550/arXiv.2310.18893},
doi = {10.48550/ARXIV.2310.18893},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2310.18893},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2425.
Zhang, Tunhou; Wen, Wei; Fedorov, Igor; Liu, Xi; Zhang, Buyun; Han, Fangqiu; Chen, Wen-Yen; Han, Yiping; Yan, Feng; Li, Hai; Chen, Yiran
DistDNAS: Search Efficient Feature Interactions within 2 Hours Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2311-00231,
title = {DistDNAS: Search Efficient Feature Interactions within 2 Hours},
author = {Tunhou Zhang and Wei Wen and Igor Fedorov and Xi Liu and Buyun Zhang and Fangqiu Han and Wen-Yen Chen and Yiping Han and Feng Yan and Hai Li and Yiran Chen},
url = {https://doi.org/10.48550/arXiv.2311.00231},
doi = {10.48550/ARXIV.2311.00231},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2311.00231},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2424.
Sun, Pingping; Fan, Shijie; Li, Shaochuan; Zhao, Yingwei; Lu, Chang; Wong, Ka-Chun; Li, Xiangtao
Automated exploitation of deep learning for cancer patient stratification across multiple types Journal Article
In: Bioinformatics, vol. 39, no. 11, pp. btad654, 2023, ISSN: 1367-4811.
@article{10.1093/bioinformatics/btad654,
title = {Automated exploitation of deep learning for cancer patient stratification across multiple types},
author = {Pingping Sun and Shijie Fan and Shaochuan Li and Yingwei Zhao and Chang Lu and Ka-Chun Wong and Xiangtao Li},
url = {https://doi.org/10.1093/bioinformatics/btad654},
doi = {10.1093/bioinformatics/btad654},
issn = {1367-4811},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Bioinformatics},
volume = {39},
number = {11},
pages = {btad654},
abstract = {Recent frameworks based on deep learning have been developed to identify cancer subtypes from high-throughput gene expression profiles. Unfortunately, the performance of deep learning is highly dependent on its neural network architectures which are often hand-crafted with expertise in deep neural networks, meanwhile, the optimization and adjustment of the network are usually costly and time consuming.To address such limitations, we proposed a fully automated deep neural architecture search model for diagnosing consensus molecular subtypes from gene expression data (DNAS). The proposed model uses ant colony algorithm, one of the heuristic swarm intelligence algorithms, to search and optimize neural network architecture, and it can automatically find the optimal deep learning model architecture for cancer diagnosis in its search space. We validated DNAS on eight colorectal cancer datasets, achieving the average accuracy of 95.48%, the average specificity of 98.07%, and the average sensitivity of 96.24%, respectively. Without the loss of generality, we investigated the general applicability of DNAS further on other cancer types from different platforms including lung cancer and breast cancer, and DNAS achieved an area under the curve of 95% and 96%, respectively. In addition, we conducted gene ontology enrichment and pathological analysis to reveal interesting insights into cancer subtype identification and characterization across multiple cancer types.The source code and data can be downloaded from https://github.com/userd113/DNAS-main. And the web server of DNAS is publicly accessible at 119.45.145.120:5001.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recent frameworks based on deep learning have been developed to identify cancer subtypes from high-throughput gene expression profiles. Unfortunately, the performance of deep learning is highly dependent on its neural network architectures which are often hand-crafted with expertise in deep neural networks, meanwhile, the optimization and adjustment of the network are usually costly and time consuming.To address such limitations, we proposed a fully automated deep neural architecture search model for diagnosing consensus molecular subtypes from gene expression data (DNAS). The proposed model uses ant colony algorithm, one of the heuristic swarm intelligence algorithms, to search and optimize neural network architecture, and it can automatically find the optimal deep learning model architecture for cancer diagnosis in its search space. We validated DNAS on eight colorectal cancer datasets, achieving the average accuracy of 95.48%, the average specificity of 98.07%, and the average sensitivity of 96.24%, respectively. Without the loss of generality, we investigated the general applicability of DNAS further on other cancer types from different platforms including lung cancer and breast cancer, and DNAS achieved an area under the curve of 95% and 96%, respectively. In addition, we conducted gene ontology enrichment and pathological analysis to reveal interesting insights into cancer subtype identification and characterization across multiple cancer types.The source code and data can be downloaded from https://github.com/userd113/DNAS-main. And the web server of DNAS is publicly accessible at 119.45.145.120:5001.
2423.
Li, Qing; Dai, Haixing; Lv, Jinglei; Zhao, Lin; Liu, Zhengliang; Wu, Zihao; Wu, Xia; Coles, Claire; Hu, Xiaoping; Liu, Tianming; Zhu, Dajiang
Individual Functional Network Abnormalities Mapping via Graph Representation-Based Neural Architecture Search Proceedings Article
In: Yang, Xiaochun; Suhartanto, Heru; Wang, Guoren; Wang, Bin; Jiang, Jing; Li, Bing; Zhu, Huaijie; Cui, Ningning (Ed.): Advanced Data Mining and Applications, pp. 79–91, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-46671-7.
@inproceedings{10.1007/978-3-031-46671-7_6,
title = {Individual Functional Network Abnormalities Mapping via Graph Representation-Based Neural Architecture Search},
author = {Qing Li and Haixing Dai and Jinglei Lv and Lin Zhao and Zhengliang Liu and Zihao Wu and Xia Wu and Claire Coles and Xiaoping Hu and Tianming Liu and Dajiang Zhu},
editor = {Xiaochun Yang and Heru Suhartanto and Guoren Wang and Bin Wang and Jing Jiang and Bing Li and Huaijie Zhu and Ningning Cui},
url = {https://link.springer.com/chapter/10.1007/978-3-031-46671-7_6},
isbn = {978-3-031-46671-7},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Advanced Data Mining and Applications},
pages = {79–91},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Prenatal alcohol exposure (PAE) has garnered increasing attention due to its detrimental effects on both neonates and expectant mothers. Recent research indicates that spatio-temporal functional brain networks (FBNs), derived from functional magnetic resonance imaging (fMRI), have the potential to reveal changes in PAE and Non-dysmorphic PAE (Non-Dys PAE) groups compared with healthy controls. However, current deep learning approaches for decomposing the FBNs are still limited to hand-crafted neural network architectures, which may not lead to optimal performance in identifying FBNs that better reveal differences between PAE and healthy controls. In this paper, we utilize a novel graph representation-based neural architecture search (GR-NAS) model to optimize the inner cell architecture of recurrent neural network (RNN) for decomposing the spatio-temporal FBNs and identifying the neuroimaging biomarkers of subtypes of PAE. Our optimized RNN cells with the GR-NAS model revealed that the functional activation decreased from healthy controls to Non-Dys PAE then to PAE groups. Our model provides a novel computational tool for the diagnosis of PAE, and uncovers the brain's functional mechanism in PAE.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Prenatal alcohol exposure (PAE) has garnered increasing attention due to its detrimental effects on both neonates and expectant mothers. Recent research indicates that spatio-temporal functional brain networks (FBNs), derived from functional magnetic resonance imaging (fMRI), have the potential to reveal changes in PAE and Non-dysmorphic PAE (Non-Dys PAE) groups compared with healthy controls. However, current deep learning approaches for decomposing the FBNs are still limited to hand-crafted neural network architectures, which may not lead to optimal performance in identifying FBNs that better reveal differences between PAE and healthy controls. In this paper, we utilize a novel graph representation-based neural architecture search (GR-NAS) model to optimize the inner cell architecture of recurrent neural network (RNN) for decomposing the spatio-temporal FBNs and identifying the neuroimaging biomarkers of subtypes of PAE. Our optimized RNN cells with the GR-NAS model revealed that the functional activation decreased from healthy controls to Non-Dys PAE then to PAE groups. Our model provides a novel computational tool for the diagnosis of PAE, and uncovers the brain's functional mechanism in PAE.
2422.
Yang, Shangshang; Zhen, Cheng; Tian, Ye; Ma, Haiping; Liu, Yuanchao; Zhang, Panpan; Zhang, Xingyi
Evolutionary Multi-Objective Neural Architecture Search for Generalized Cognitive Diagnosis Models Proceedings Article
In: 2023 5th International Conference on Data-driven Optimization of Complex Systems (DOCS), pp. 1-10, 2023.
@inproceedings{10294588,
title = {Evolutionary Multi-Objective Neural Architecture Search for Generalized Cognitive Diagnosis Models},
author = {Shangshang Yang and Cheng Zhen and Ye Tian and Haiping Ma and Yuanchao Liu and Panpan Zhang and Xingyi Zhang},
url = {https://ieeexplore.ieee.org/abstract/document/10294588},
doi = {10.1109/DOCS60977.2023.10294588},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 5th International Conference on Data-driven Optimization of Complex Systems (DOCS)},
pages = {1-10},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2421.
Shi, Huihong; Xu, Yang; Wang, Yuefei; Mao, Wendong; Wang, Zhongfeng
NASA-F: FPGA-Oriented Search and Acceleration for Multiplication-Reduced Hybrid Networks Journal Article
In: IEEE Transactions on Circuits and Systems I: Regular Papers, pp. 1-14, 2023.
@article{10308526,
title = {NASA-F: FPGA-Oriented Search and Acceleration for Multiplication-Reduced Hybrid Networks},
author = {Huihong Shi and Yang Xu and Yuefei Wang and Wendong Mao and Zhongfeng Wang},
doi = {10.1109/TCSI.2023.3327965},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
pages = {1-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2420.
Dooley, Samuel; Sukthanker, Rhea Sanjay; Dickerson, John P; White, Colin; Hutter, Frank; Goldblum, Micah
Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition Proceedings Article
In: Thirty-seventh Conference on Neural Information Processing Systems, 2023.
@inproceedings{<LineBreak>dooley2023rethinking,
title = {Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition},
author = {Samuel Dooley and Rhea Sanjay Sukthanker and John P Dickerson and Colin White and Frank Hutter and Micah Goldblum},
url = {https://openreview.net/forum?id=1vzF4zWQ1E},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Thirty-seventh Conference on Neural Information Processing Systems},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2419.
Niu, Ruicheng; Zhu, Ziyuan; Leng, Tao; Li, Chaofei; Liu, Yuxin; Meng, Dan
Improving Adversarial Robustness via Channel and Depth Compatibility Proceedings Article
In: Yang, Xiaochun; Suhartanto, Heru; Wang, Guoren; Wang, Bin; Jiang, Jing; Li, Bing; Zhu, Huaijie; Cui, Ningning (Ed.): Advanced Data Mining and Applications, pp. 137–149, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-46677-9.
@inproceedings{10.1007/978-3-031-46677-9_10,
title = {Improving Adversarial Robustness via Channel and Depth Compatibility},
author = {Ruicheng Niu and Ziyuan Zhu and Tao Leng and Chaofei Li and Yuxin Liu and Dan Meng},
editor = {Xiaochun Yang and Heru Suhartanto and Guoren Wang and Bin Wang and Jing Jiang and Bing Li and Huaijie Zhu and Ningning Cui},
isbn = {978-3-031-46677-9},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Advanced Data Mining and Applications},
pages = {137–149},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Several deep neural networks are vulnerable to adversarial samples that are imperceptible to humans. To address this challenge, a range of techniques have been proposed to design more robust model architectures. However, previous research has primarily focused on identifying atomic structures that are more resilient, while our work focuses on adapting the model in two spatial dimensions: width and depth. In this paper, we present a multi-objective neural architecture search (NAS) method that searches for optimal widths for different layers in spatial dimensions, referred to as DW-Net. We also propose a novel adversarial sample generation technique for one-shot that enhances search space diversity and promotes search efficiency. Our experimental results demonstrate that the proposed optimal neural architecture outperforms state-of-the-art NAS-based networks widely used in the literature in terms of adversarial accuracy, under different adversarial attacks and for different-sized tasks.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Several deep neural networks are vulnerable to adversarial samples that are imperceptible to humans. To address this challenge, a range of techniques have been proposed to design more robust model architectures. However, previous research has primarily focused on identifying atomic structures that are more resilient, while our work focuses on adapting the model in two spatial dimensions: width and depth. In this paper, we present a multi-objective neural architecture search (NAS) method that searches for optimal widths for different layers in spatial dimensions, referred to as DW-Net. We also propose a novel adversarial sample generation technique for one-shot that enhances search space diversity and promotes search efficiency. Our experimental results demonstrate that the proposed optimal neural architecture outperforms state-of-the-art NAS-based networks widely used in the literature in terms of adversarial accuracy, under different adversarial attacks and for different-sized tasks.
2418.
Passos, Dário; Mishra, Puneet
Deep Tutti Frutti: Exploring CNN architectures for dry matter prediction in fruit from multi-fruit near-infrared spectra Journal Article
In: Chemometrics and Intelligent Laboratory Systems, vol. 243, pp. 105023, 2023, ISSN: 0169-7439.
@article{PASSOS2023105023,
title = {Deep Tutti Frutti: Exploring CNN architectures for dry matter prediction in fruit from multi-fruit near-infrared spectra},
author = {Dário Passos and Puneet Mishra},
url = {https://www.sciencedirect.com/science/article/pii/S0169743923002733},
doi = {https://doi.org/10.1016/j.chemolab.2023.105023},
issn = {0169-7439},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Chemometrics and Intelligent Laboratory Systems},
volume = {243},
pages = {105023},
abstract = {Convolutional Neural Networks (CNNs) have proven to be a valuable Deep Learning (DL) algorithm to model near-infrared spectral data in Chemometrics. However, optimizing CNN architectures and their associated hyperparameters for specific tasks is challenging. In this study, we explore the development of 1D-CNN architectures for the task of fruit dry matter (DM) estimation, testing various designs and optimization strategies to achieve a generic DL model that is robust against data fluctuations. The models are built using a multi-fruit data set (n = 2397) that includes NIR spectra of apples, kiwis, mangoes, and pears. The obtained CNN models are compared with PLS (taken as baseline), and to Locally Weighted PLS (LW-PLS) models. In general, the optimized CNN architectures obtained lower RMSEs (best RMSE = 0.605 %) than PLS (RMSE = 0.892 %) and LW-PLS (RMSE = 0.687 %) on a holdout test set. For this specific task, CNNs start outperforming PLS when the number of training samples is around 500. Furthermore, it is also shown how a global CNN model, trained on multi-fruit data, performs against PLS models of individual fruits in the sub-tasks of individual fruit DM prediction and generalization to an external mango data set. Overall, with proper architecture optimization, CNNs show strong performance and generalization for NIR-based dry matter estimation across diverse fruits.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Convolutional Neural Networks (CNNs) have proven to be a valuable Deep Learning (DL) algorithm to model near-infrared spectral data in Chemometrics. However, optimizing CNN architectures and their associated hyperparameters for specific tasks is challenging. In this study, we explore the development of 1D-CNN architectures for the task of fruit dry matter (DM) estimation, testing various designs and optimization strategies to achieve a generic DL model that is robust against data fluctuations. The models are built using a multi-fruit data set (n = 2397) that includes NIR spectra of apples, kiwis, mangoes, and pears. The obtained CNN models are compared with PLS (taken as baseline), and to Locally Weighted PLS (LW-PLS) models. In general, the optimized CNN architectures obtained lower RMSEs (best RMSE = 0.605 %) than PLS (RMSE = 0.892 %) and LW-PLS (RMSE = 0.687 %) on a holdout test set. For this specific task, CNNs start outperforming PLS when the number of training samples is around 500. Furthermore, it is also shown how a global CNN model, trained on multi-fruit data, performs against PLS models of individual fruits in the sub-tasks of individual fruit DM prediction and generalization to an external mango data set. Overall, with proper architecture optimization, CNNs show strong performance and generalization for NIR-based dry matter estimation across diverse fruits.
2417.
Schrodi, Simon; Stoll, Danny; Ru, Binxin; Sukthanker, Rhea; Brox, Thomas; Hutter, Frank
Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars Proceedings Article
In: Advances in Neural Information Processing Systems, 2023.
@inproceedings{schrodi2023construction,
title = {Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars},
author = {Simon Schrodi and Danny Stoll and Binxin Ru and Rhea Sukthanker and Thomas Brox and Frank Hutter},
url = {https://openreview.net/pdf?id=Hpt1i5j6wh},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Advances in Neural Information Processing Systems},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2416.
Sinha, Nilotpal; Shabayek, Abd El Rahman; Kacem, Anis; Rostami, Peyman; Shneider, Carl; Aouada, Djamila
Hardware Aware Evolutionary Neural Architecture Search using Representation Similarity Metric Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2311-03923,
title = {Hardware Aware Evolutionary Neural Architecture Search using Representation Similarity Metric},
author = {Nilotpal Sinha and Abd El Rahman Shabayek and Anis Kacem and Peyman Rostami and Carl Shneider and Djamila Aouada},
url = {https://doi.org/10.48550/arXiv.2311.03923},
doi = {10.48550/ARXIV.2311.03923},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2311.03923},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2415.
McKnight, Shaun; MacKinnon, Christopher; Pierce, S. Gareth; Mohseni, Ehsan; Tunukovic, Vedran; MacLeod, Charles N.; Vithanage, Randika K. W.; OHare, Tom
3-Dimensional residual neural architecture search for ultrasonic defect detection Technical Report
2023.
@techreport{mcknight20233dimensional,
title = {3-Dimensional residual neural architecture search for ultrasonic defect detection},
author = {Shaun McKnight and Christopher MacKinnon and S. Gareth Pierce and Ehsan Mohseni and Vedran Tunukovic and Charles N. MacLeod and Randika K. W. Vithanage and Tom OHare},
url = {https://arxiv.org/abs/2311.01867},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2414.
Li, Yueyang; Zhang, Qican
NAS-assisted single-shot real-time 3D measurement Proceedings Article
In: Shao, Xiaopeng (Ed.): Third International Computing Imaging Conference (CITA 2023), pp. 129210S, International Society for Optics and Photonics SPIE, 2023.
@inproceedings{10.1117/12.2688044,
title = {NAS-assisted single-shot real-time 3D measurement},
author = {Yueyang Li and Qican Zhang},
editor = {Xiaopeng Shao},
url = {https://doi.org/10.1117/12.2688044},
doi = {10.1117/12.2688044},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Third International Computing Imaging Conference (CITA 2023)},
volume = {12921},
pages = {129210S},
publisher = {SPIE},
organization = {International Society for Optics and Photonics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2413.
Prezja, Fabi; Annala, Leevi; Kiiskinen, Sampsa; Lahtinen, Suvi; Ojala, Timo
2023.
@techreport{unknownb,
title = {Adaptive Variance Thresholding: A Novel Approach to Improve Existing Deep Transfer Vision Models and Advance Automatic Knee-Joint Osteoarthritis Classification},
author = {Fabi Prezja and Leevi Annala and Sampsa Kiiskinen and Suvi Lahtinen and Timo Ojala},
url = {https://www.researchgate.net/profile/Fabi-Prezja/publication/375336306_Adaptive_Variance_Thresholding_A_Novel_Approach_to_Improve_Existing_Deep_Transfer_Vision_Models_and_Advance_Automatic_Knee-Joint_Osteoarthritis_Classification/links/6546cda8ce88b87031c4e5ad/Adaptive-Variance-Thresholding-A-Novel-Approach-to-Improve-Existing-Deep-Transfer-Vision-Models-and-Advance-Automatic-Knee-Joint-Osteoarthritis-Classification.pdf},
doi = {10.13140/RG.2.2.10736.02566},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2412.
Hou, Jia; Zhang, Jingyu; Chen, Qi; Xiang, Siwei; Meng, Yishuo; Wang, Jianfei; Lu, Cimang; Yang, Chen
In: Information, vol. 14, no. 11, 2023, ISSN: 2078-2489.
@article{info14110604,
title = {POSS-CNN: An Automatically Generated Convolutional Neural Network with Precision and Operation Separable Structure Aiming at Target Recognition and Detection},
author = {Jia Hou and Jingyu Zhang and Qi Chen and Siwei Xiang and Yishuo Meng and Jianfei Wang and Cimang Lu and Chen Yang},
url = {https://www.mdpi.com/2078-2489/14/11/604},
doi = {10.3390/info14110604},
issn = {2078-2489},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Information},
volume = {14},
number = {11},
abstract = {Artificial intelligence is changing and influencing our world. As one of the main algorithms in the field of artificial intelligence, convolutional neural networks (CNNs) have developed rapidly in recent years. Especially after the emergence of NASNet, CNNs have gradually pushed the idea of AutoML to the public’s attention, and large numbers of new structures designed by automatic searches are appearing. These networks are usually based on reinforcement learning and evolutionary learning algorithms. However, sometimes, the blocks of these networks are complex, and there is no small model for simpler tasks. Therefore, this paper proposes POSS-CNN aiming at target recognition and detection, which employs a multi-branch CNN structure with PSNC and a method of automatic parallel selection for super parameters based on a multi-branch CNN structure. Moreover, POSS-CNN can be broken up. By choosing a single branch or the combination of two branches as the “benchmark”, as well as the overall POSS-CNN, we can achieve seven models with different precision and operations. The test accuracy of POSS-CNN for a recognition task tested on a CIFAR10 dataset can reach 86.4%, which is equivalent to AlexNet and VggNet, but the operation and parameters of the whole model in this paper are 45.9% and 45.8% of AlexNet, and 29.5% and 29.4% of VggNet. The mAP of POSS-CNN for a detection task tested on the LSVH dataset is 45.8, inferior to the 62.3 of YOLOv3. However, compared with YOLOv3, the operation and parameters of the model in this paper are reduced by 57.4% and 15.6%, respectively. After being accelerated by WRA, POSS-CNN for a detection task tested on an LSVH dataset can achieve 27 fps, and the energy efficiency is 0.42 J/f, which is 5 times and 96.6 times better than GPU 2080Ti in performance and energy efficiency, respectively.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Artificial intelligence is changing and influencing our world. As one of the main algorithms in the field of artificial intelligence, convolutional neural networks (CNNs) have developed rapidly in recent years. Especially after the emergence of NASNet, CNNs have gradually pushed the idea of AutoML to the public’s attention, and large numbers of new structures designed by automatic searches are appearing. These networks are usually based on reinforcement learning and evolutionary learning algorithms. However, sometimes, the blocks of these networks are complex, and there is no small model for simpler tasks. Therefore, this paper proposes POSS-CNN aiming at target recognition and detection, which employs a multi-branch CNN structure with PSNC and a method of automatic parallel selection for super parameters based on a multi-branch CNN structure. Moreover, POSS-CNN can be broken up. By choosing a single branch or the combination of two branches as the “benchmark”, as well as the overall POSS-CNN, we can achieve seven models with different precision and operations. The test accuracy of POSS-CNN for a recognition task tested on a CIFAR10 dataset can reach 86.4%, which is equivalent to AlexNet and VggNet, but the operation and parameters of the whole model in this paper are 45.9% and 45.8% of AlexNet, and 29.5% and 29.4% of VggNet. The mAP of POSS-CNN for a detection task tested on the LSVH dataset is 45.8, inferior to the 62.3 of YOLOv3. However, compared with YOLOv3, the operation and parameters of the model in this paper are reduced by 57.4% and 15.6%, respectively. After being accelerated by WRA, POSS-CNN for a detection task tested on an LSVH dataset can achieve 27 fps, and the energy efficiency is 0.42 J/f, which is 5 times and 96.6 times better than GPU 2080Ti in performance and energy efficiency, respectively.
2411.
Wu, Meng-Ting; Tsai, Chun-Wei
Training-free neural architecture search: A review Journal Article
In: ICT Express, 2023, ISSN: 2405-9595.
@article{WU2023,
title = {Training-free neural architecture search: A review},
author = {Meng-Ting Wu and Chun-Wei Tsai},
url = {https://www.sciencedirect.com/science/article/pii/S2405959523001443},
doi = {https://doi.org/10.1016/j.icte.2023.11.001},
issn = {2405-9595},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {ICT Express},
abstract = {The goal of neural architecture search (NAS) is to either downsize the neural architecture and model of a deep neural network (DNN), adjust a neural architecture to improve its end result, or even speed up the whole training process. Such improvements make it possible to generate or install the model of a DNN on a small device, such as a device of internet of things or wireless sensor network. Because most NAS algorithms are time-consuming, finding out a way to reduce their computation costs has now become a critical research issue. The training-free method (also called the zero-shot learning) provides an alternative way to estimate how good a neural architecture is more efficiently during the process of NAS by using a lightweight score function instead of a general training process to avoid incurring heavy costs. This paper starts with a brief discussion of DNN and NAS, followed by a brief review of both model-dependent and model-independent training-free score functions. A brief introduction to the search algorithms and benchmarks that were widely used in a training-free NAS will also be given in this paper. The changes, potential, open issues, and future trends of this research topic are then addressed in the end of this paper.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The goal of neural architecture search (NAS) is to either downsize the neural architecture and model of a deep neural network (DNN), adjust a neural architecture to improve its end result, or even speed up the whole training process. Such improvements make it possible to generate or install the model of a DNN on a small device, such as a device of internet of things or wireless sensor network. Because most NAS algorithms are time-consuming, finding out a way to reduce their computation costs has now become a critical research issue. The training-free method (also called the zero-shot learning) provides an alternative way to estimate how good a neural architecture is more efficiently during the process of NAS by using a lightweight score function instead of a general training process to avoid incurring heavy costs. This paper starts with a brief discussion of DNN and NAS, followed by a brief review of both model-dependent and model-independent training-free score functions. A brief introduction to the search algorithms and benchmarks that were widely used in a training-free NAS will also be given in this paper. The changes, potential, open issues, and future trends of this research topic are then addressed in the end of this paper.
2410.
Dong, Dong; Jiang, Hongxu; Wei, Xuekai; Song, Yanfei; Zhuang, Xu; Wang, Jason
ETNAS: An energy consumption task-driven neural architecture search Journal Article
In: Sustainable Computing: Informatics and Systems, vol. 40, pp. 100926, 2023, ISSN: 2210-5379.
@article{DONG2023100926,
title = {ETNAS: An energy consumption task-driven neural architecture search},
author = {Dong Dong and Hongxu Jiang and Xuekai Wei and Yanfei Song and Xu Zhuang and Jason Wang},
url = {https://www.sciencedirect.com/science/article/pii/S2210537923000811},
doi = {https://doi.org/10.1016/j.suscom.2023.100926},
issn = {2210-5379},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Sustainable Computing: Informatics and Systems},
volume = {40},
pages = {100926},
abstract = {Neural Architecture Search (NAS) is crucial in the field of sustainable computing as it facilitates the development of highly efficient and effective neural networks. However, it cannot automate the deployment of neural networks to accommodate specific hardware resources and task requirements. This paper introduces ETNAS, which is a hardware-aware multi-objective optimal neural network architecture search algorithm based on the differentiable neural network architecture search method (DARTS). The algorithm searches for a lower-power neural network architecture with guaranteed inference accuracy by modifying the loss function of the differentiable neural network architecture search. We modify the dense network in DARTS to simultaneously search for networks with a lower memory footprint, enabling them to run on memory-constrained edge-end devices. We collected data on the power consumption and time consumption of numerous common operators on FPGA and Domain-Specific Architectures (DSA). The experimental results demonstrate that ETNAS achieves comparable accuracy performance and time efficiency while consuming less power compared to state-of-the-art algorithms, thereby validating its effectiveness in practical applications and contributing to the reduction of carbon emissions in intelligent cyber–physical systems (ICPS) edge computing inference.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Architecture Search (NAS) is crucial in the field of sustainable computing as it facilitates the development of highly efficient and effective neural networks. However, it cannot automate the deployment of neural networks to accommodate specific hardware resources and task requirements. This paper introduces ETNAS, which is a hardware-aware multi-objective optimal neural network architecture search algorithm based on the differentiable neural network architecture search method (DARTS). The algorithm searches for a lower-power neural network architecture with guaranteed inference accuracy by modifying the loss function of the differentiable neural network architecture search. We modify the dense network in DARTS to simultaneously search for networks with a lower memory footprint, enabling them to run on memory-constrained edge-end devices. We collected data on the power consumption and time consumption of numerous common operators on FPGA and Domain-Specific Architectures (DSA). The experimental results demonstrate that ETNAS achieves comparable accuracy performance and time efficiency while consuming less power compared to state-of-the-art algorithms, thereby validating its effectiveness in practical applications and contributing to the reduction of carbon emissions in intelligent cyber–physical systems (ICPS) edge computing inference.
2409.
Tang, Siao; Wang, Xin; Chen, Hong; Guan, Chaoyu; Tang, Yansong; Zhu, Wenwu
Lightweight Diffusion Models with Distillation-Based Block Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2311-04950,
title = {Lightweight Diffusion Models with Distillation-Based Block Neural Architecture Search},
author = {Siao Tang and Xin Wang and Hong Chen and Chaoyu Guan and Yansong Tang and Wenwu Zhu},
url = {https://doi.org/10.48550/arXiv.2311.04950},
doi = {10.48550/ARXIV.2311.04950},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2311.04950},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2408.
Li, Yuke; Baik, Jiwon; Rahman, Md Marufi; Anagnostopoulos, Iraklis; Li, Ruopu; Shu, Tong
Pareto Optimization of CNN Models via Hardware-Aware Neural Architecture Search for Drainage Crossing Classification on Resource-Limited Devices Proceedings Article
In: Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, pp. 1767–1775, Association for Computing Machinery, Denver, CO, USA, 2023, ISBN: 9798400707858.
@inproceedings{10.1145/3624062.3624258,
title = {Pareto Optimization of CNN Models via Hardware-Aware Neural Architecture Search for Drainage Crossing Classification on Resource-Limited Devices},
author = {Yuke Li and Jiwon Baik and Md Marufi Rahman and Iraklis Anagnostopoulos and Ruopu Li and Tong Shu},
url = {https://doi.org/10.1145/3624062.3624258},
doi = {10.1145/3624062.3624258},
isbn = {9798400707858},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis},
pages = {1767–1775},
publisher = {Association for Computing Machinery},
address = {Denver, CO, USA},
series = {SC-W '23},
abstract = {Embedded devices, constrained by limited memory and processors, require deep learning models to be tailored to their specifications. This research explores customized model architectures for classifying drainage crossing images. Building on the foundational ResNet-18, this paper aims to maximize prediction accuracy, reduce memory size, and minimize inference latency. Various configurations were systematically probed by leveraging hardware-aware neural architecture search, accumulating 1,717 experimental results over six benchmarking variants. The experimental data analysis, enhanced by nn-Meter, provided a comprehensive understanding of inference latency across four different predictors. Significantly, a Pareto front analysis with three objectives of accuracy, latency, and memory resulted in five non-dominated solutions. These standout models showcased efficiency while retaining accuracy, offering a compelling alternative to the conventional ResNet-18 when deployed in resource-constrained environments. The paper concludes by highlighting insights drawn from the results and suggesting avenues for future exploration.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Embedded devices, constrained by limited memory and processors, require deep learning models to be tailored to their specifications. This research explores customized model architectures for classifying drainage crossing images. Building on the foundational ResNet-18, this paper aims to maximize prediction accuracy, reduce memory size, and minimize inference latency. Various configurations were systematically probed by leveraging hardware-aware neural architecture search, accumulating 1,717 experimental results over six benchmarking variants. The experimental data analysis, enhanced by nn-Meter, provided a comprehensive understanding of inference latency across four different predictors. Significantly, a Pareto front analysis with three objectives of accuracy, latency, and memory resulted in five non-dominated solutions. These standout models showcased efficiency while retaining accuracy, offering a compelling alternative to the conventional ResNet-18 when deployed in resource-constrained environments. The paper concludes by highlighting insights drawn from the results and suggesting avenues for future exploration.
2407.
Hsieh, Jun-Wei; Chou, Cheng-Han; Chang, Ming-Ching; Chen, Ping-Yang; Santra, Santanu; Huang, Chih-Sheng
Mean-Shift Based Differentiable Architecture Search Journal Article
In: IEEE Transactions on Artificial Intelligence, pp. 1-11, 2023.
@article{10310657,
title = {Mean-Shift Based Differentiable Architecture Search},
author = {Jun-Wei Hsieh and Cheng-Han Chou and Ming-Ching Chang and Ping-Yang Chen and Santanu Santra and Chih-Sheng Huang},
url = {https://ieeexplore.ieee.org/abstract/document/10310657},
doi = {10.1109/TAI.2023.3329792},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Artificial Intelligence},
pages = {1-11},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2406.
Chang, Chi-Chih; Sung, Yuan-Yao; Yu, Shixing; Huang, Ning-Chi; Marculescu, Diana; Wu, Kai-Chiang
FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformer Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2311-03912,
title = {FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformer},
author = {Chi-Chih Chang and Yuan-Yao Sung and Shixing Yu and Ning-Chi Huang and Diana Marculescu and Kai-Chiang Wu},
url = {https://doi.org/10.48550/arXiv.2311.03912},
doi = {10.48550/ARXIV.2311.03912},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2311.03912},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2405.
Chang, Chen-Chia; Pan, Jingyu; Xie, Zhiyao; Zhang, Tunhou; Hu, Jiang; Chen, Yiran
Towards Fully Automated Machine Learning for Routability Estimator Development Journal Article
In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2023.
@article{10310247,
title = {Towards Fully Automated Machine Learning for Routability Estimator Development},
author = {Chen-Chia Chang and Jingyu Pan and Zhiyao Xie and Tunhou Zhang and Jiang Hu and Yiran Chen},
url = {https://ieeexplore.ieee.org/abstract/document/10310247},
doi = {10.1109/TCAD.2023.3330818},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2404.
An, Taegun; Joo, Changhee
CycleGANAS: Differentiable Neural Architecture Search for CycleGAN Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2311-07162,
title = {CycleGANAS: Differentiable Neural Architecture Search for CycleGAN},
author = {Taegun An and Changhee Joo},
url = {https://doi.org/10.48550/arXiv.2311.07162},
doi = {10.48550/ARXIV.2311.07162},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2311.07162},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2403.
Ji, Junzhong; Wang, Xingyu
Convolutional architecture search based on particle swarm algorithm for functional brain network classification Journal Article
In: Applied Soft Computing, vol. 149, pp. 111049, 2023, ISSN: 1568-4946.
@article{JI2023111049,
title = {Convolutional architecture search based on particle swarm algorithm for functional brain network classification},
author = {Junzhong Ji and Xingyu Wang},
url = {https://www.sciencedirect.com/science/article/pii/S1568494623010670},
doi = {https://doi.org/10.1016/j.asoc.2023.111049},
issn = {1568-4946},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Applied Soft Computing},
volume = {149},
pages = {111049},
abstract = {The functional brain network (FBN) classification based on convolutional neural networks (CNN) is of great significance for discovery and diagnosis of brain diseases, and has attracted increasing attention. However, all the CNN architectures of current studies mainly depend on hand-crafted, which are labor-intensive and unreliable. To solve it, we propose a neural architecture search (NAS) method based on particle swarm optimization, to automatically design the CNN architecture for FBN classification. Specifically, this method includes three phases, namely the individual expression phase, the individual evaluation phase, and the individual update phase. In the first phase, we treat the neural architecture as the individual in particle swarm. The individual vector consists of six elements, and the value of each element represents the number of a special convolution operation. The six special convolution operations can effectively extract brain network multilevel topological features. In the second phase, we propose a novel surrogate-assisted predictor to evaluate the fitness of the individuals more efficiently. In the last phase, we apply the predicted fitness to acquire the historical optimum of each individual and the global optimum of the population, and use them to update all individuals in the particle swarm. The second and third phases are repeatedly performed until the end condition is met. Experiments on benchmark datasets demonstrate that the CNN architecture searched by our method achieves better classification performance than state-of-the-art hand-crafted CNN architectures.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The functional brain network (FBN) classification based on convolutional neural networks (CNN) is of great significance for discovery and diagnosis of brain diseases, and has attracted increasing attention. However, all the CNN architectures of current studies mainly depend on hand-crafted, which are labor-intensive and unreliable. To solve it, we propose a neural architecture search (NAS) method based on particle swarm optimization, to automatically design the CNN architecture for FBN classification. Specifically, this method includes three phases, namely the individual expression phase, the individual evaluation phase, and the individual update phase. In the first phase, we treat the neural architecture as the individual in particle swarm. The individual vector consists of six elements, and the value of each element represents the number of a special convolution operation. The six special convolution operations can effectively extract brain network multilevel topological features. In the second phase, we propose a novel surrogate-assisted predictor to evaluate the fitness of the individuals more efficiently. In the last phase, we apply the predicted fitness to acquire the historical optimum of each individual and the global optimum of the population, and use them to update all individuals in the particle swarm. The second and third phases are repeatedly performed until the end condition is met. Experiments on benchmark datasets demonstrate that the CNN architecture searched by our method achieves better classification performance than state-of-the-art hand-crafted CNN architectures.