Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
2768.
Fan, Yicheng; Alon, Dana; Shen, Jingyue; Peng, Daiyi; Kumar, Keshav; Long, Yun; Wang, Xin; Iliopoulos, Fotis; Juan, Da-Cheng; Vee, Erik
LayerNAS: Neural Architecture Search in Polynomial Complexity Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11517,
title = {LayerNAS: Neural Architecture Search in Polynomial Complexity},
author = {Yicheng Fan and Dana Alon and Jingyue Shen and Daiyi Peng and Keshav Kumar and Yun Long and Xin Wang and Fotis Iliopoulos and Da-Cheng Juan and Erik Vee},
url = {https://doi.org/10.48550/arXiv.2304.11517},
doi = {10.48550/arXiv.2304.11517},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11517},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2767.
Wang, Di; Du, Bo; Zhang, Liangpei; Tao, Dacheng
HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11701,
title = {HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search},
author = {Di Wang and Bo Du and Liangpei Zhang and Dacheng Tao},
url = {https://doi.org/10.48550/arXiv.2304.11701},
doi = {10.48550/arXiv.2304.11701},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11701},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2766.
Li, Chao; Xu, Hao; He, Kun
Meta-multigraph Search: Rethinking Meta-structure on Heterogeneous Information Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11574,
title = {Meta-multigraph Search: Rethinking Meta-structure on Heterogeneous Information Networks},
author = {Chao Li and Hao Xu and Kun He},
url = {https://doi.org/10.48550/arXiv.2304.11574},
doi = {10.48550/arXiv.2304.11574},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11574},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2765.
Fu, Yonggan; Li, Yuecheng; Li, Chenghui; Saragih, Jason M.; Zhang, Peizhao; Dai, Xiaoliang; Lin, Yingyan
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-11835,
title = {Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence},
author = {Yonggan Fu and Yuecheng Li and Chenghui Li and Jason M. Saragih and Peizhao Zhang and Xiaoliang Dai and Yingyan Lin},
url = {https://doi.org/10.48550/arXiv.2304.11835},
doi = {10.48550/arXiv.2304.11835},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.11835},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2764.
Li, YuFei; Wu, Jia; Deng, TianJin
Meta-GNAS: Meta-reinforcement learning for graph neural architecture search Journal Article
In: Engineering Applications of Artificial Intelligence, vol. 123, pp. 106300, 2023, ISSN: 0952-1976.
@article{LI2023106300,
title = {Meta-GNAS: Meta-reinforcement learning for graph neural architecture search},
author = {YuFei Li and Jia Wu and TianJin Deng},
url = {https://www.sciencedirect.com/science/article/pii/S0952197623004840},
doi = {https://doi.org/10.1016/j.engappai.2023.106300},
issn = {0952-1976},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Engineering Applications of Artificial Intelligence},
volume = {123},
pages = {106300},
abstract = {Graph Neural architecture search (GNAS) has shown great success in designing many prominent models on non-Euclidean data. However, the existing GNAS methods need to search from scratch on new tasks, which is time-consuming and inefficient in real application scenarios. In this paper, we propose a meta-reinforcement learning method for Graph Neural Architecture Search (Meta-GNAS) to improve the learning efficiency on new tasks by leveraging the knowledge learned from previous tasks. As far as we know, it is the first work that applies meta-learning to GNAS tasks. Moreover, to further improve the efficiency in tackling a new task, we use a predictive model to evaluate the accuracy of sampled graph neural architecture, instead of training it from scratch. The experiment results demonstrate that the architecture designed by Meta-GNAS outperforms the state-of-art manually designed architectures, and the search speed is faster than other search methods, with an average search time of fewer than 210 GPU seconds on 6 datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Graph Neural architecture search (GNAS) has shown great success in designing many prominent models on non-Euclidean data. However, the existing GNAS methods need to search from scratch on new tasks, which is time-consuming and inefficient in real application scenarios. In this paper, we propose a meta-reinforcement learning method for Graph Neural Architecture Search (Meta-GNAS) to improve the learning efficiency on new tasks by leveraging the knowledge learned from previous tasks. As far as we know, it is the first work that applies meta-learning to GNAS tasks. Moreover, to further improve the efficiency in tackling a new task, we use a predictive model to evaluate the accuracy of sampled graph neural architecture, instead of training it from scratch. The experiment results demonstrate that the architecture designed by Meta-GNAS outperforms the state-of-art manually designed architectures, and the search speed is faster than other search methods, with an average search time of fewer than 210 GPU seconds on 6 datasets.
2763.
Xiao, Anqi; Shen, Biluo; Tian, Jie; Hu, Zhenhua
PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks Journal Article
In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-13, 2023.
@article{10109323,
title = {PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks},
author = {Anqi Xiao and Biluo Shen and Jie Tian and Zhenhua Hu},
doi = {10.1109/TNNLS.2023.3264551},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2762.
Liu, Yong; Zhou, Xin; Zhong, Wei
Multi-Modality Image Fusion and Object Detection Based on Semantic Information Journal Article
In: Entropy, vol. 25, no. 5, 2023, ISSN: 1099-4300.
@article{e25050718,
title = {Multi-Modality Image Fusion and Object Detection Based on Semantic Information},
author = {Yong Liu and Xin Zhou and Wei Zhong},
url = {https://www.mdpi.com/1099-4300/25/5/718},
doi = {10.3390/e25050718},
issn = {1099-4300},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Entropy},
volume = {25},
number = {5},
abstract = {Infrared and visible image fusion (IVIF) aims to provide informative images by combining complementary information from different sensors. Existing IVIF methods based on deep learning focus on strengthening the network with increasing depth but often ignore the importance of transmission characteristics, resulting in the degradation of important information. In addition, while many methods use various loss functions or fusion rules to retain complementary features of both modes, the fusion results often retain redundant or even invalid information.In order to accurately extract the effective information from both infrared images and visible light images without omission or redundancy, and to better serve downstream tasks such as target detection with the fused image, we propose a multi-level structure search attention fusion network based on semantic information guidance, which realizes the fusion of infrared and visible images in an end-to-end way. Our network has two main contributions: the use of neural architecture search (NAS) and the newly designed multilevel adaptive attention module (MAAB). These methods enable our network to retain the typical characteristics of the two modes while removing useless information for the detection task in the fusion results. In addition, our loss function and joint training method can establish a reliable relationship between the fusion network and subsequent detection tasks. Extensive experiments on the new dataset (M3FD) show that our fusion method has achieved advanced performance in both subjective and objective evaluations, and the mAP in the object detection task is improved by 0.5% compared to the second-best method (FusionGAN).},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Infrared and visible image fusion (IVIF) aims to provide informative images by combining complementary information from different sensors. Existing IVIF methods based on deep learning focus on strengthening the network with increasing depth but often ignore the importance of transmission characteristics, resulting in the degradation of important information. In addition, while many methods use various loss functions or fusion rules to retain complementary features of both modes, the fusion results often retain redundant or even invalid information.In order to accurately extract the effective information from both infrared images and visible light images without omission or redundancy, and to better serve downstream tasks such as target detection with the fused image, we propose a multi-level structure search attention fusion network based on semantic information guidance, which realizes the fusion of infrared and visible images in an end-to-end way. Our network has two main contributions: the use of neural architecture search (NAS) and the newly designed multilevel adaptive attention module (MAAB). These methods enable our network to retain the typical characteristics of the two modes while removing useless information for the detection task in the fusion results. In addition, our loss function and joint training method can establish a reliable relationship between the fusion network and subsequent detection tasks. Extensive experiments on the new dataset (M3FD) show that our fusion method has achieved advanced performance in both subjective and objective evaluations, and the mAP in the object detection task is improved by 0.5% compared to the second-best method (FusionGAN).
2761.
Gao, Jianliang; Chen, Jiamin; Oloulade, Babatounde Moctard; Al-Sabri, Raeed; Lyu, Tengfei; Zhang, Ji; Li, Zhao
CommGNAS: Unsupervised Graph Neural Architecture Search for Community Detection Journal Article
In: IEEE Transactions on Emerging Topics in Computing, pp. 1-12, 2023.
@article{10112632,
title = {CommGNAS: Unsupervised Graph Neural Architecture Search for Community Detection},
author = {Jianliang Gao and Jiamin Chen and Babatounde Moctard Oloulade and Raeed Al-Sabri and Tengfei Lyu and Ji Zhang and Zhao Li},
doi = {10.1109/TETC.2023.3270181},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Emerging Topics in Computing},
pages = {1-12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2760.
Bria, Alessandro; Ciccio, Paolo De; DÁlessandro, Tiziana; Fontanella, Francesco
Ä Novel Evolutionary Approach for Neural Architecture Search Proceedings Article
In: Stefano, Claudio De; Fontanella, Francesco; Vanneschi, Leonardo (Ed.): Ärtificial Life and Evolutionary Computation", pp. 195–204, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-31183-3.
@inproceedings{10.1007/978-3-031-31183-3_16,
title = {Ä Novel Evolutionary Approach for Neural Architecture Search},
author = {Alessandro Bria and Paolo De Ciccio and Tiziana DÁlessandro and Francesco Fontanella},
editor = {Claudio De Stefano and Francesco Fontanella and Leonardo Vanneschi},
url = {https://link.springer.com/chapter/10.1007/978-3-031-31183-3_16},
isbn = {978-3-031-31183-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ärtificial Life and Evolutionary Computation"},
pages = {195--204},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Convolutional Neural Networks (CNNs) have proven to be an effective tool in many real-world applications. The main problem of CNNs is the lack of a well-defined and largely shared set of criteria for the choice of architecture for a given problem. This lack represents a drawback for this approach since the choice of architecture plays a crucial role in CNNs'performance. Usually, these architectures are manually designed by experts. However, such a design process is computationally intensive because of the trial-and-error process and also not easy to realize due to the high level of expertise required. Recently, to try to overcome those drawbacks, many techniques that automize the task of designing the architecture neural networks have been proposed. To denote these techniques has been defined the term ``Neural Architecture Search'' (NAS). Among the many methods available for NAS, Evolutionary Computation (EC) methods have recently gained much attention and success. In this paper, we present a novel approach based on evolutionary computation to optimize CNNs. The proposed approach is based on a newly devised structure which encodes both hyperparameters and the architecture of a CNN. The experimental results show that the proposed approach allows us to achieve better performance than that achieved by state-of-the-art CNNs on a real-world problem. Furthermore, the proposed approach can generate smaller networks than the state-of-the-art CNNs used for the comparison.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Convolutional Neural Networks (CNNs) have proven to be an effective tool in many real-world applications. The main problem of CNNs is the lack of a well-defined and largely shared set of criteria for the choice of architecture for a given problem. This lack represents a drawback for this approach since the choice of architecture plays a crucial role in CNNs'performance. Usually, these architectures are manually designed by experts. However, such a design process is computationally intensive because of the trial-and-error process and also not easy to realize due to the high level of expertise required. Recently, to try to overcome those drawbacks, many techniques that automize the task of designing the architecture neural networks have been proposed. To denote these techniques has been defined the term ``Neural Architecture Search'' (NAS). Among the many methods available for NAS, Evolutionary Computation (EC) methods have recently gained much attention and success. In this paper, we present a novel approach based on evolutionary computation to optimize CNNs. The proposed approach is based on a newly devised structure which encodes both hyperparameters and the architecture of a CNN. The experimental results show that the proposed approach allows us to achieve better performance than that achieved by state-of-the-art CNNs on a real-world problem. Furthermore, the proposed approach can generate smaller networks than the state-of-the-art CNNs used for the comparison.
2759.
Ao, Lei; Feng, Kaiyuan; Sheng, Kai; Zhao, Hongyu; He, Xin; Chen, Zigang
TPENAS: A Two-Phase Evolutionary Neural Architecture Search for Remote Sensing Image Classification Journal Article
In: Remote Sensing, vol. 15, no. 8, 2023, ISSN: 2072-4292.
@article{rs15082212,
title = {TPENAS: A Two-Phase Evolutionary Neural Architecture Search for Remote Sensing Image Classification},
author = {Lei Ao and Kaiyuan Feng and Kai Sheng and Hongyu Zhao and Xin He and Zigang Chen},
url = {https://www.mdpi.com/2072-4292/15/8/2212},
doi = {10.3390/rs15082212},
issn = {2072-4292},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Remote Sensing},
volume = {15},
number = {8},
abstract = {The application of deep learning in remote sensing image classification has been paid more and more attention by industry and academia. However, manually designed remote sensing image classification models based on convolutional neural networks usually require sophisticated expert knowledge. Moreover, it is notoriously difficult to design a model with both high classification accuracy and few parameters. Recently, neural architecture search (NAS) has emerged as an effective method that can greatly reduce the heavy burden of manually designing models. However, it remains a challenge to search for a classification model with high classification accuracy and few parameters in the huge search space. To tackle this challenge, we propose TPENAS, a two-phase evolutionary neural architecture search framework, which optimizes the model using computational intelligence techniques in two search phases. In the first search phase, TPENAS searches for the optimal depth of the model. In the second search phase, TPENAS searches for the structure of the model from the perspective of the whole model. Experiments on three open benchmark datasets demonstrate that our proposed TPENAS outperforms the state-of-the-art baselines in both classification accuracy and reducing parameters.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The application of deep learning in remote sensing image classification has been paid more and more attention by industry and academia. However, manually designed remote sensing image classification models based on convolutional neural networks usually require sophisticated expert knowledge. Moreover, it is notoriously difficult to design a model with both high classification accuracy and few parameters. Recently, neural architecture search (NAS) has emerged as an effective method that can greatly reduce the heavy burden of manually designing models. However, it remains a challenge to search for a classification model with high classification accuracy and few parameters in the huge search space. To tackle this challenge, we propose TPENAS, a two-phase evolutionary neural architecture search framework, which optimizes the model using computational intelligence techniques in two search phases. In the first search phase, TPENAS searches for the optimal depth of the model. In the second search phase, TPENAS searches for the structure of the model from the perspective of the whole model. Experiments on three open benchmark datasets demonstrate that our proposed TPENAS outperforms the state-of-the-art baselines in both classification accuracy and reducing parameters.
2758.
Wei, He; Yang, Yuekui; Wu, Haiyang; Tang, Yangyang; Liu, Meixi; Li, Jianfeng
Automatic Feature Selection By One-Shot Neural Architecture Search In Recommendation Systems Proceedings Article
In: Proceedings of the ACM Web Conference 2023, pp. 1765–1772, Association for Computing Machinery, Austin, TX, USA, 2023, ISBN: 9781450394161.
@inproceedings{10.1145/3543507.3583444,
title = {Automatic Feature Selection By One-Shot Neural Architecture Search In Recommendation Systems},
author = {He Wei and Yuekui Yang and Haiyang Wu and Yangyang Tang and Meixi Liu and Jianfeng Li},
url = {https://doi.org/10.1145/3543507.3583444},
doi = {10.1145/3543507.3583444},
isbn = {9781450394161},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the ACM Web Conference 2023},
pages = {1765–1772},
publisher = {Association for Computing Machinery},
address = {Austin, TX, USA},
series = {WWW '23},
abstract = {Feature selection is crucial in large-scale recommendation system, which can not only reduce the computational cost, but also improve the recommendation efficiency. Most existing works rank the features and then select the top-k ones as the final feature subset. However, they assess feature importance individually and ignore the interrelationship between features. Consequently, multiple features with high relevance may be selected simultaneously, resulting in sub-optimal result. In this work, we solve this problem by proposing an AutoML-based feature selection framework that can automatically search the optimal feature subset. Specifically, we first embed the search space into a weight-sharing Supernet. Then, a two-stage neural architecture search method is employed to evaluate the feature quality. In the first stage, a well-designed sampling method considering feature convergence fairness is applied to train the Supernet. In the second stage, a reinforcement learning method is used to search for the optimal feature subset efficiently. The Experimental results on two real datasets demonstrate the superior performance of new framework over other solutions. Our proposed method obtain significant improvement with a 20% reduction in the amount of features on the Criteo. More validation experiments demonstrate the ability and robustness of the framework.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Feature selection is crucial in large-scale recommendation system, which can not only reduce the computational cost, but also improve the recommendation efficiency. Most existing works rank the features and then select the top-k ones as the final feature subset. However, they assess feature importance individually and ignore the interrelationship between features. Consequently, multiple features with high relevance may be selected simultaneously, resulting in sub-optimal result. In this work, we solve this problem by proposing an AutoML-based feature selection framework that can automatically search the optimal feature subset. Specifically, we first embed the search space into a weight-sharing Supernet. Then, a two-stage neural architecture search method is employed to evaluate the feature quality. In the first stage, a well-designed sampling method considering feature convergence fairness is applied to train the Supernet. In the second stage, a reinforcement learning method is used to search for the optimal feature subset efficiently. The Experimental results on two real datasets demonstrate the superior performance of new framework over other solutions. Our proposed method obtain significant improvement with a 20% reduction in the amount of features on the Criteo. More validation experiments demonstrate the ability and robustness of the framework.
2757.
Wu, Shiguang; Wang, Yaqing; Jing, Qinghe; Dong, Daxiang; Dou, Dejing; Yao, Quanming
ColdNAS: Search to Modulate for User Cold-Start Recommendation Proceedings Article
In: Proceedings of the ACM Web Conference 2023, pp. 1021–1031, Association for Computing Machinery, Austin, TX, USA, 2023, ISBN: 9781450394161.
@inproceedings{10.1145/3543507.3583344,
title = {ColdNAS: Search to Modulate for User Cold-Start Recommendation},
author = {Shiguang Wu and Yaqing Wang and Qinghe Jing and Daxiang Dong and Dejing Dou and Quanming Yao},
url = {https://doi.org/10.1145/3543507.3583344},
doi = {10.1145/3543507.3583344},
isbn = {9781450394161},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the ACM Web Conference 2023},
pages = {1021–1031},
publisher = {Association for Computing Machinery},
address = {Austin, TX, USA},
series = {WWW '23},
abstract = {Making personalized recommendation for cold-start users, who only have a few interaction histories, is a challenging problem in recommendation systems. Recent works leverage hypernetworks to directly map user interaction histories to user-specific parameters, which are then used to modulate predictor by feature-wise linear modulation function. These works obtain the state-of-the-art performance. However, the physical meaning of scaling and shifting in recommendation data is unclear. Instead of using a fixed modulation function and deciding modulation position by expertise, we propose a modulation framework called ColdNAS for user cold-start problem, where we look for proper modulation structure, including function and position, via neural architecture search. We design a search space which covers broad models and theoretically prove that this search space can be transformed to a much smaller space, enabling an efficient and robust one-shot search algorithm. Extensive experimental results on benchmark datasets show that ColdNAS consistently performs the best. We observe that different modulation functions lead to the best performance on different datasets, which validates the necessity of designing a searching-based method. Codes are available at https://github.com/LARS-research/ColdNAS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Making personalized recommendation for cold-start users, who only have a few interaction histories, is a challenging problem in recommendation systems. Recent works leverage hypernetworks to directly map user interaction histories to user-specific parameters, which are then used to modulate predictor by feature-wise linear modulation function. These works obtain the state-of-the-art performance. However, the physical meaning of scaling and shifting in recommendation data is unclear. Instead of using a fixed modulation function and deciding modulation position by expertise, we propose a modulation framework called ColdNAS for user cold-start problem, where we look for proper modulation structure, including function and position, via neural architecture search. We design a search space which covers broad models and theoretically prove that this search space can be transformed to a much smaller space, enabling an efficient and robust one-shot search algorithm. Extensive experimental results on benchmark datasets show that ColdNAS consistently performs the best. We observe that different modulation functions lead to the best performance on different datasets, which validates the necessity of designing a searching-based method. Codes are available at https://github.com/LARS-research/ColdNAS.
2756.
Xie, Xiangning; Sun, Yanan; Liu, Yuqiao; Zhang, Mengjie; Tan, Kay Chen
Architecture Augmentation for Performance Predictor via Graph Isomorphism Journal Article
In: IEEE Transactions on Cybernetics, pp. 1-13, 2023.
@article{10109990,
title = {Architecture Augmentation for Performance Predictor via Graph Isomorphism},
author = {Xiangning Xie and Yanan Sun and Yuqiao Liu and Mengjie Zhang and Kay Chen Tan},
doi = {10.1109/TCYB.2023.3267109},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Cybernetics},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2755.
Mansoori, Mohammad Amir; Casu, Mario R.
Multi-objective Framework for Training and Hardware Co-optimization in FPGAs Proceedings Article
In: Berta, Riccardo; Gloria, Alessandro De (Ed.): Äpplications in Electronics Pervading Industry, Environment and Society", pp. 273–278, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-30333-3.
@inproceedings{10.1007/978-3-031-30333-3_36,
title = {Multi-objective Framework for Training and Hardware Co-optimization in FPGAs},
author = {Mohammad Amir Mansoori and Mario R. Casu},
editor = {Riccardo Berta and Alessandro De Gloria},
url = {https://link.springer.com/chapter/10.1007/978-3-031-30333-3_36},
isbn = {978-3-031-30333-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Äpplications in Electronics Pervading Industry, Environment and Society"},
pages = {273--278},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Älthough several works have recently addressed the problem of performance co-optimization for hardware and network training for Convolutional Neural Networks, most of them considered either a fixed network or a given hardware architecture. In this work, we propose a new framework for joint optimization of network architecture and hardware configurations based on Bayesian Optimization (BO) on top of High Level Synthesis. The multi-objective nature of this framework allows for the definition of various hardware and network performance goals as well as multiple constraints, and the multi-objective BO allows to easily obtain a set of Pareto points. We evaluate our methodology on a network optimized for an FPGA target and show that the Pareto set obtained by the proposed joint-optimization outperforms other methods based on a separate optimization or random search."},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Älthough several works have recently addressed the problem of performance co-optimization for hardware and network training for Convolutional Neural Networks, most of them considered either a fixed network or a given hardware architecture. In this work, we propose a new framework for joint optimization of network architecture and hardware configurations based on Bayesian Optimization (BO) on top of High Level Synthesis. The multi-objective nature of this framework allows for the definition of various hardware and network performance goals as well as multiple constraints, and the multi-objective BO allows to easily obtain a set of Pareto points. We evaluate our methodology on a network optimized for an FPGA target and show that the Pareto set obtained by the proposed joint-optimization outperforms other methods based on a separate optimization or random search."
2754.
Guo, Xinwei; Wu, Yong; Miao, Jingjing; Chen, Yang
LiteGaze: Neural architecture search for efficient gaze estimation Journal Article
In: PLOS ONE, vol. 18, no. 5, pp. 1-11, 2023.
@article{10.1371/journal.pone.0284814,
title = {LiteGaze: Neural architecture search for efficient gaze estimation},
author = {Xinwei Guo and Yong Wu and Jingjing Miao and Yang Chen},
url = {https://doi.org/10.1371/journal.pone.0284814},
doi = {10.1371/journal.pone.0284814},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {PLOS ONE},
volume = {18},
number = {5},
pages = {1-11},
publisher = {Public Library of Science},
abstract = {Gaze estimation plays a critical role in human-centered vision applications such as human–computer interaction and virtual reality. Although significant progress has been made in automatic gaze estimation by deep convolutional neural networks, it is still difficult to directly deploy deep learning based gaze estimation models across different edge devices, due to the high computational cost and various resource constraints. This work proposes LiteGaze, a deep learning framework to learn architectures for efficient gaze estimation via neural architecture search (NAS). Inspired by the once-for-all model (Cai et al., 2020), this work decouples the model training and architecture search into two different stages. In particular, a supernet is trained to support diverse architectural settings. Then specialized sub-networks are selected from the obtained supernet, given different efficiency constraints. Extensive experiments are performed on two gaze estimation datasets and demonstrate the superiority of the proposed method over previous works, advancing the real-time gaze estimation on edge devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Gaze estimation plays a critical role in human-centered vision applications such as human–computer interaction and virtual reality. Although significant progress has been made in automatic gaze estimation by deep convolutional neural networks, it is still difficult to directly deploy deep learning based gaze estimation models across different edge devices, due to the high computational cost and various resource constraints. This work proposes LiteGaze, a deep learning framework to learn architectures for efficient gaze estimation via neural architecture search (NAS). Inspired by the once-for-all model (Cai et al., 2020), this work decouples the model training and architecture search into two different stages. In particular, a supernet is trained to support diverse architectural settings. Then specialized sub-networks are selected from the obtained supernet, given different efficiency constraints. Extensive experiments are performed on two gaze estimation datasets and demonstrate the superiority of the proposed method over previous works, advancing the real-time gaze estimation on edge devices.
2753.
Wang, Haibin; Ge, Ce; Chen, Hesen; Sun, Xiuyu
PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2304-14636,
title = {PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search},
author = {Haibin Wang and Ce Ge and Hesen Chen and Xiuyu Sun},
url = {https://doi.org/10.48550/arXiv.2304.14636},
doi = {10.48550/arXiv.2304.14636},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2304.14636},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2752.
Singh, Bhavneet; Mansukhani, Jai
Comparing Neural Architectures to Find the Best Model Suited for Edge Devices Proceedings Article
In: Sharma, Harish; Saha, Apu Kumar; Prasad, Mukesh (Ed.): Proceedings of International Conference on Intelligent Vision and Computing (ICIVC 2022), pp. 195–203, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-31164-2.
@inproceedings{10.1007/978-3-031-31164-2_16,
title = {Comparing Neural Architectures to Find the Best Model Suited for Edge Devices},
author = {Bhavneet Singh and Jai Mansukhani},
editor = {Harish Sharma and Apu Kumar Saha and Mukesh Prasad},
url = {https://link.springer.com/chapter/10.1007/978-3-031-31164-2_16},
isbn = {978-3-031-31164-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of International Conference on Intelligent Vision and Computing (ICIVC 2022)},
pages = {195--203},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Training large-scale neural network models is computationally expensive and demands a great deal of resources. It is an important area of study with a lot of potential for the future of the AI industry. In recent years, the power of computer hardware has significantly improved and we have new breakthroughs in deep learning. With these innovations, the computational cost of training large neural network models has declined by at least 10 folds in high- and average-performance machines. In this research, we explore NAL, AutoML, and other frameworks to determine the best suitable model for edge devices. The biggest improvements compared to reference models can be acquired if the NAS algorithm is co-designed with the corresponding inference engine.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Training large-scale neural network models is computationally expensive and demands a great deal of resources. It is an important area of study with a lot of potential for the future of the AI industry. In recent years, the power of computer hardware has significantly improved and we have new breakthroughs in deep learning. With these innovations, the computational cost of training large neural network models has declined by at least 10 folds in high- and average-performance machines. In this research, we explore NAL, AutoML, and other frameworks to determine the best suitable model for edge devices. The biggest improvements compared to reference models can be acquired if the NAS algorithm is co-designed with the corresponding inference engine.
2751.
Mlodozeniec, Bruno; Reisser, Matthias; Louizos, Christos
Hyperparameter Optimization through Neural Network Partitioning Proceedings Article
In: ICLR 2023, 2023.
@inproceedings{DBLP:journals/corr/abs-2304-14766,
title = {Hyperparameter Optimization through Neural Network Partitioning},
author = {Bruno Mlodozeniec and Matthias Reisser and Christos Louizos},
url = {https://doi.org/10.48550/arXiv.2304.14766},
doi = {10.48550/arXiv.2304.14766},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = { ICLR 2023},
journal = { ICLR 2023},
volume = {abs/2304.14766},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2750.
Garcia-Garcia, Cosijopii; Morales-Reyes, Alicia; Escalante, Hugo
Continuous Cartesian Genetic Programming based representation for Multi-Objective Neural Architecture Search Technical Report
2023.
@techreport{unknown,
title = {Continuous Cartesian Genetic Programming based representation for Multi-Objective Neural Architecture Search},
author = {Cosijopii Garcia-Garcia and Alicia Morales-Reyes and Hugo Escalante},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2749.
Chen, Yu; Yang, Mingyu; Kim, Hun-Seok
Search for Efficient Deep Visual-Inertial Odometry Through Neural Architecture Search Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10095166,
title = {Search for Efficient Deep Visual-Inertial Odometry Through Neural Architecture Search},
author = {Yu Chen and Mingyu Yang and Hun-Seok Kim},
url = {https://ieeexplore.ieee.org/abstract/document/10095166},
doi = {10.1109/ICASSP49357.2023.10095166},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2748.
Keserwani, Prateek; Miriyala, Srinivas Soumitri; Rajendiran, Vikram N.; Shivamurthappa, Pradeep N.
Receptive Field Reliant Zero-Cost Proxies for Neural Architecture Search Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10096753,
title = {Receptive Field Reliant Zero-Cost Proxies for Neural Architecture Search},
author = {Prateek Keserwani and Srinivas Soumitri Miriyala and Vikram N. Rajendiran and Pradeep N. Shivamurthappa},
url = {https://ieeexplore.ieee.org/abstract/document/10096621},
doi = {10.1109/ICASSP49357.2023.10096753},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2747.
Rumiantsev, Pavel; Coates, Mark
Performing Neural Architecture Search Without Gradients Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10094582,
title = {Performing Neural Architecture Search Without Gradients},
author = {Pavel Rumiantsev and Mark Coates},
url = {https://ieeexplore.ieee.org/abstract/document/10094582},
doi = {10.1109/ICASSP49357.2023.10094582},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2746.
Fan, Yi; Niu, Zhong-Han; Yang, Yu-Bin
Data-Aware Zero-Shot Neural Architecture Search for Image Recognition Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10094741,
title = {Data-Aware Zero-Shot Neural Architecture Search for Image Recognition},
author = {Yi Fan and Zhong-Han Niu and Yu-Bin Yang},
url = {https://ieeexplore.ieee.org/abstract/document/10094741},
doi = {10.1109/ICASSP49357.2023.10094741},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2745.
Xie, Lunchen; Huang, Kaiyu; Xu, Fan; Shi, Qingjiang
ZO-DARTS: Differentiable Architecture Search with Zeroth-Order Approximation Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10096612,
title = {ZO-DARTS: Differentiable Architecture Search with Zeroth-Order Approximation},
author = {Lunchen Xie and Kaiyu Huang and Fan Xu and Qingjiang Shi},
url = {https://ieeexplore.ieee.org/abstract/document/10096612},
doi = {10.1109/ICASSP49357.2023.10096612},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2744.
Cai, Zicheng; Chen, Lei; Liu, Hai-Lin
BHE-DARTS: Bilevel Optimization Based on Hypergradient Estimation for Differentiable Architecture Search Proceedings Article
In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023.
@inproceedings{10095940,
title = {BHE-DARTS: Bilevel Optimization Based on Hypergradient Estimation for Differentiable Architecture Search},
author = {Zicheng Cai and Lei Chen and Hai-Lin Liu},
url = {https://ieeexplore.ieee.org/abstract/document/10095940},
doi = {10.1109/ICASSP49357.2023.10095940},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2743.
Jeong, Joonhyun; Yu, Joonsang; Park, Geondo; Han, Dongyoon; Yoo, Youngjoon
GeNAS: Neural Architecture Search with Better Generalization Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-08611,
title = {GeNAS: Neural Architecture Search with Better Generalization},
author = {Joonhyun Jeong and Joonsang Yu and Geondo Park and Dongyoon Han and Youngjoon Yoo},
url = {https://doi.org/10.48550/arXiv.2305.08611},
doi = {10.48550/arXiv.2305.08611},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.08611},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2742.
Zhang, Jianwei; Zhang, Lei; Li, Dong
A Unified Search Framework for Data Augmentation and Neural Architecture on Small-scale Image Datasets Journal Article
In: IEEE Transactions on Cognitive and Developmental Systems, pp. 1-1, 2023.
@article{10124814,
title = {A Unified Search Framework for Data Augmentation and Neural Architecture on Small-scale Image Datasets},
author = {Jianwei Zhang and Lei Zhang and Dong Li},
url = {https://ieeexplore.ieee.org/abstract/document/10124814},
doi = {10.1109/TCDS.2023.3274177},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Cognitive and Developmental Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2741.
Sun, Jialiang; Yao, Wen; Jiang, Tingsong; Chen, Xiaoqian
Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-07308,
title = {Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation},
author = {Jialiang Sun and Wen Yao and Tingsong Jiang and Xiaoqian Chen},
url = {https://doi.org/10.48550/arXiv.2305.07308},
doi = {10.48550/arXiv.2305.07308},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.07308},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2740.
Li, Guihong; Bhardwaj, Kartikeya; Yang, Yuedong; Marculescu, Radu
TIPS: Topologically Important Path Sampling for Anytime Neural Networks Journal Article
In: CoRR, vol. abs/2305.08021, 2023.
@article{DBLP:journals/corr/abs-2305-08021,
title = {TIPS: Topologically Important Path Sampling for Anytime Neural Networks},
author = {Guihong Li and Kartikeya Bhardwaj and Yuedong Yang and Radu Marculescu},
url = {https://doi.org/10.48550/arXiv.2305.08021},
doi = {10.48550/arXiv.2305.08021},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.08021},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2739.
Hadi, Russul H.; Hady, Haider N.; Hasan, Ahmed M.; Al-Jodah, Ammar; Humaidi, Amjad J.
Improved Fault Classification for Predictive Maintenance in Industrial IoT Based on AutoML: A Case Study of Ball-Bearing Faults Journal Article
In: Processes, vol. 11, no. 5, 2023, ISSN: 2227-9717.
@article{pr11051507,
title = {Improved Fault Classification for Predictive Maintenance in Industrial IoT Based on AutoML: A Case Study of Ball-Bearing Faults},
author = {Russul H. Hadi and Haider N. Hady and Ahmed M. Hasan and Ammar Al-Jodah and Amjad J. Humaidi},
url = {https://www.mdpi.com/2227-9717/11/5/1507},
doi = {10.3390/pr11051507},
issn = {2227-9717},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Processes},
volume = {11},
number = {5},
abstract = {The growing complexity of data derived from Industrial Internet of Things (IIoT) systems presents substantial challenges for traditional machine-learning techniques, which struggle to effectively manage the needs of predictive maintenance applications. Automated machine-learning (AutoML) techniques present a promising solution by streamlining the machine-learning process, reducing the necessity for manual hyperparameter tuning and computational resources, thereby positioning themselves as a potentially transformative innovation in the Industry 4.0 era. This research introduces two distinct models: AutoML, employing PyCaret, and Auto Deep Neural Network (AutoDNN), utilizing AutoKeras, both aimed at accurately identifying various types of faults in ball bearings. The proposed models were evaluated using the Case Western Reserve University (CWRU) bearing faults dataset, and the results showed a notable performance in terms of achieving high accuracy, recall, precision, and F1 score on the testing and validation sets. Compared to recent studies, the proposed AutoML models demonstrated superior performance, surpassing alternative approaches even when they utilized a larger number of features, thus highlighting the effectiveness of the proposed methodology. This research offers valuable insights for those interested in harnessing the potential of AutoML techniques in IIoT applications, with implications for industries such as manufacturing and energy. By automating the machine-learning process, AutoML models can help decrease the time and cost related to predictive maintenance, which is crucial for industries where unplanned downtime can lead to substantial financial losses.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The growing complexity of data derived from Industrial Internet of Things (IIoT) systems presents substantial challenges for traditional machine-learning techniques, which struggle to effectively manage the needs of predictive maintenance applications. Automated machine-learning (AutoML) techniques present a promising solution by streamlining the machine-learning process, reducing the necessity for manual hyperparameter tuning and computational resources, thereby positioning themselves as a potentially transformative innovation in the Industry 4.0 era. This research introduces two distinct models: AutoML, employing PyCaret, and Auto Deep Neural Network (AutoDNN), utilizing AutoKeras, both aimed at accurately identifying various types of faults in ball bearings. The proposed models were evaluated using the Case Western Reserve University (CWRU) bearing faults dataset, and the results showed a notable performance in terms of achieving high accuracy, recall, precision, and F1 score on the testing and validation sets. Compared to recent studies, the proposed AutoML models demonstrated superior performance, surpassing alternative approaches even when they utilized a larger number of features, thus highlighting the effectiveness of the proposed methodology. This research offers valuable insights for those interested in harnessing the potential of AutoML techniques in IIoT applications, with implications for industries such as manufacturing and energy. By automating the machine-learning process, AutoML models can help decrease the time and cost related to predictive maintenance, which is crucial for industries where unplanned downtime can lead to substantial financial losses.
2738.
Wang, Yifan; Zhong, Linlin
NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs Technical Report
2023.
@techreport{wang2023naspinn,
title = {NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs},
author = {Yifan Wang and Linlin Zhong},
url = {https://arxiv.org/abs/2305.10127},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2737.
Benmeziane, Hadjer; Lammie, Corey; Boybat, Irem; Rasch, Malte; Gallo, Manuel Le; Tsai, Hsinyu; Muralidhar, Ramachandran; Niar, Smail; Hamza, Ouarnoughi; Narayanan, Vijay; Sebastian, Abu; Maghraoui, Kaoutar El
AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing Technical Report
2023.
@techreport{benmeziane2023analognas,
title = {AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing},
author = {Hadjer Benmeziane and Corey Lammie and Irem Boybat and Malte Rasch and Manuel Le Gallo and Hsinyu Tsai and Ramachandran Muralidhar and Smail Niar and Ouarnoughi Hamza and Vijay Narayanan and Abu Sebastian and Kaoutar El Maghraoui},
url = {https://arxiv.org/abs/2305.10459},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2736.
Li, Yanjing; Xu, Sheng; Cao, Xianbin; Zhuo, Lián; Zhang, Baochang; Wang, Tian; Guo, Guodong
DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs Technical Report
2023.
@techreport{li2023dcpnas,
title = {DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs},
author = {Yanjing Li and Sheng Xu and Xianbin Cao and Lián Zhuo and Baochang Zhang and Tian Wang and Guodong Guo},
url = {https://arxiv.org/abs/2306.15390},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2735.
Wang, Wei; Wang, Xianpeng; Song, Xiangman
Adaptive lightweight convolutional neural architecture search for segmentation problem Journal Article
In: Engineering Optimization, vol. 0, no. 0, pp. 1-18, 2023.
@article{doi:10.1080/0305215X.2023.2225036,
title = {Adaptive lightweight convolutional neural architecture search for segmentation problem},
author = {Wei Wang and Xianpeng Wang and Xiangman Song},
url = {https://doi.org/10.1080/0305215X.2023.2225036},
doi = {10.1080/0305215X.2023.2225036},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Engineering Optimization},
volume = {0},
number = {0},
pages = {1-18},
publisher = {Taylor & Francis},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2734.
Peng, Hongwu; Zhou, Shanglin; Luo, Yukui; Xu, Nuo; Duan, Shijin; Ran, Ran; Zhao, Jiahui; Wang, Chenghong; Geng, Tong; Wen, Wujie; Xu, Xiaolin; Ding, Caiwen
PASNet: Polynomial Architecture Search Framework for Two-party Computation-based Secure Neural Network Deployment Technical Report
2023.
@techreport{peng2023pasnet,
title = {PASNet: Polynomial Architecture Search Framework for Two-party Computation-based Secure Neural Network Deployment},
author = {Hongwu Peng and Shanglin Zhou and Yukui Luo and Nuo Xu and Shijin Duan and Ran Ran and Jiahui Zhao and Chenghong Wang and Tong Geng and Wujie Wen and Xiaolin Xu and Caiwen Ding},
url = {https://arxiv.org/abs/2306.15513},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2733.
Lacharme, Guillaume; Cardot, Hubert; Lenté, Christophe; Monmarché, Nicolas
DARTS with Degeneracy Correction Proceedings Article
In: Pertusa, Antonio; Gallego, Antonio Javier; Sánchez, Joan Andreu; Domingues, Inês (Ed.): Pattern Recognition and Image Analysis, pp. 40–53, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-36616-1.
@inproceedings{10.1007/978-3-031-36616-1_4,
title = {DARTS with Degeneracy Correction},
author = {Guillaume Lacharme and Hubert Cardot and Christophe Lenté and Nicolas Monmarché},
editor = {Antonio Pertusa and Antonio Javier Gallego and Joan Andreu Sánchez and Inês Domingues},
url = {https://link.springer.com/chapter/10.1007/978-3-031-36616-1_4},
isbn = {978-3-031-36616-1},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Pattern Recognition and Image Analysis},
pages = {40–53},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {The neural architecture search (NAS) is characterized by a wide search space and a time consuming objective function. Many papers have dealt with the reduction of the cost of the objective function assessment. Among them, there is DARTS paper [1] that proposes to transform the original discrete problem into a continuous one. This paper builds an overparameterized network called hypernetwork and weights its edges by continuous coefficients. This approach allows to considerably reduce the computational cost, but the quality of the obtained architectures is highly variable. We propose to reduce this variability by introducing a convex depth regularization. We also add a heuristic that controls the number of unweighted operations. The goal is to correct the short term bias, introduced by the hypergradient approximation. Finally, we will show the efficiency of these proposals by starting again the work developed in Dots paper [2].},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The neural architecture search (NAS) is characterized by a wide search space and a time consuming objective function. Many papers have dealt with the reduction of the cost of the objective function assessment. Among them, there is DARTS paper [1] that proposes to transform the original discrete problem into a continuous one. This paper builds an overparameterized network called hypernetwork and weights its edges by continuous coefficients. This approach allows to considerably reduce the computational cost, but the quality of the obtained architectures is highly variable. We propose to reduce this variability by introducing a convex depth regularization. We also add a heuristic that controls the number of unweighted operations. The goal is to correct the short term bias, introduced by the hypergradient approximation. Finally, we will show the efficiency of these proposals by starting again the work developed in Dots paper [2].
2732.
Zhang, Zijun; Lamson, Adam R.; Shelley, Michael; Troyanskaya, Olga G.
Interpretable neural architecture search and transfer learning for understanding sequence dependent enzymatic reactions Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-11917,
title = {Interpretable neural architecture search and transfer learning for understanding sequence dependent enzymatic reactions},
author = {Zijun Zhang and Adam R. Lamson and Michael Shelley and Olga G. Troyanskaya},
url = {https://doi.org/10.48550/arXiv.2305.11917},
doi = {10.48550/arXiv.2305.11917},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.11917},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2731.
Liu, Zhu; Liu, Jinyuan; Wu, Guanyao; Fan, Xin; Liu, Risheng
Embracing Compact and Robust Architectures for Multi-Exposure Image Fusion Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-12236,
title = {Embracing Compact and Robust Architectures for Multi-Exposure Image Fusion},
author = {Zhu Liu and Jinyuan Liu and Guanyao Wu and Xin Fan and Risheng Liu},
url = {https://doi.org/10.48550/arXiv.2305.12236},
doi = {10.48550/arXiv.2305.12236},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.12236},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2730.
Zhang, Ya-Lin; Zhou, Jun; Ren, Yankun; Zhang, Yue; Yang, Xinxing; Li, Meng; Shi, Qitao; Li, Longfei
ALT: An Automatic System for Long Tail Scenario Modeling Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-11390,
title = {ALT: An Automatic System for Long Tail Scenario Modeling},
author = {Ya-Lin Zhang and Jun Zhou and Yankun Ren and Yue Zhang and Xinxing Yang and Meng Li and Qitao Shi and Longfei Li},
url = {https://doi.org/10.48550/arXiv.2305.11390},
doi = {10.48550/arXiv.2305.11390},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.11390},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2729.
Xu, Peng; Zhang, Lin; Liu, Xuanzhou; Sun, Jiaqi; Zhao, Yue; Yang, Haiqing; Yu, Bei
Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-14065,
title = {Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks},
author = {Peng Xu and Lin Zhang and Xuanzhou Liu and Jiaqi Sun and Yue Zhao and Haiqing Yang and Bei Yu},
url = {https://doi.org/10.48550/arXiv.2305.14065},
doi = {10.48550/arXiv.2305.14065},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.14065},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2728.
Ye, Zhen; Xue, Wei; Tan, Xu; Liu, Qifeng; Guo, Yike
NAS-FM: Neural Architecture Search for Tunable and Interpretable Sound Synthesis based on Frequency Modulation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-12868,
title = {NAS-FM: Neural Architecture Search for Tunable and Interpretable Sound Synthesis based on Frequency Modulation},
author = {Zhen Ye and Wei Xue and Xu Tan and Qifeng Liu and Yike Guo},
url = {https://doi.org/10.48550/arXiv.2305.12868},
doi = {10.48550/arXiv.2305.12868},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.12868},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2727.
Yin, Kangyong; Huang, Zhechen; Qiu, Suo; Wang, Zhi; Tao, Fengbo; Liang, Wei; Huang, Haosheng
An action recognition method based on neural architecture search Proceedings Article
In: Wang, Hongzhi; Kong, Xiangjie (Ed.): International Conference on Internet of Things and Machine Learning (IoTML 2022), pp. 126401Q, International Society for Optics and Photonics SPIE, 2023.
@inproceedings{10.1117/12.2673723,
title = {An action recognition method based on neural architecture search},
author = {Kangyong Yin and Zhechen Huang and Suo Qiu and Zhi Wang and Fengbo Tao and Wei Liang and Haosheng Huang},
editor = {Hongzhi Wang and Xiangjie Kong},
url = {https://doi.org/10.1117/12.2673723},
doi = {10.1117/12.2673723},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {International Conference on Internet of Things and Machine Learning (IoTML 2022)},
volume = {12640},
pages = {126401Q},
publisher = {SPIE},
organization = {International Society for Optics and Photonics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2726.
Rajapakshe, Thejan; Rana, Rajib; Khalifa, Sara; Sisman, Berrak; Schuller, Björn W.
Improving Speech Emotion Recognition Performance using Differentiable Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-14402,
title = {Improving Speech Emotion Recognition Performance using Differentiable Architecture Search},
author = {Thejan Rajapakshe and Rajib Rana and Sara Khalifa and Berrak Sisman and Björn W. Schuller},
url = {https://doi.org/10.48550/arXiv.2305.14402},
doi = {10.48550/arXiv.2305.14402},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.14402},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2725.
Davy, N.; Waheed, U. Bin; Koeshidayatullah, A.; El-Husseiny, A.
Automated Deep Learning (AutoDL) for Facies Prediction: Implementation and Strategy Journal Article
In: vol. 2023, no. 1, pp. 1-5, 2023, ISSN: 2214-4609.
@article{eage:/content/papers/10.3997/2214-4609.202310455,
title = {Automated Deep Learning (AutoDL) for Facies Prediction: Implementation and Strategy},
author = {N. Davy and U. Bin Waheed and A. Koeshidayatullah and A. El-Husseiny},
url = {https://www.earthdoc.org/content/papers/10.3997/2214-4609.202310455},
doi = {https://doi.org/10.3997/2214-4609.202310455},
issn = {2214-4609},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
volume = {2023},
number = {1},
pages = {1-5},
publisher = {Ëuropean Association of Geoscientists & Engineers},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2724.
Deutel, Mark; Kontes, Georgios D.; Mutschler, Christopher; Teich, Jürgen
Augmented Random Search for Multi-Objective Bayesian Optimization of Neural Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-14109,
title = {Augmented Random Search for Multi-Objective Bayesian Optimization of Neural Networks},
author = {Mark Deutel and Georgios D. Kontes and Christopher Mutschler and Jürgen Teich},
url = {https://doi.org/10.48550/arXiv.2305.14109},
doi = {10.48550/arXiv.2305.14109},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.14109},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2723.
Wei, Wei; Zhao, Shuyi; Xu, Songzheng; Zhang, Lei; Zhang, Yanning
Semi-Supervised Neural Architecture Search for Hyperspectral Imagery Classification Method With Dynamic Feature Clustering Journal Article
In: IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1-14, 2023.
@article{10132869,
title = {Semi-Supervised Neural Architecture Search for Hyperspectral Imagery Classification Method With Dynamic Feature Clustering},
author = {Wei Wei and Shuyi Zhao and Songzheng Xu and Lei Zhang and Yanning Zhang},
url = {https://ieeexplore.ieee.org/abstract/document/10132869},
doi = {10.1109/TGRS.2023.3279437},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Geoscience and Remote Sensing},
volume = {61},
pages = {1-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2722.
Zhang, Jiuling; Ding, Zhiming
Small Temperature is All You Need for Differentiable Architecture Search Proceedings Article
In: Kashima, Hisashi; Ide, Tsuyoshi; Peng, Wen-Chih (Ed.): Ädvances in Knowledge Discovery and Data Mining", pp. 303–315, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-33374-3.
@inproceedings{10.1007/978-3-031-33374-3_24,
title = {Small Temperature is All You Need for Differentiable Architecture Search},
author = {Jiuling Zhang and Zhiming Ding},
editor = {Hisashi Kashima and Tsuyoshi Ide and Wen-Chih Peng},
isbn = {978-3-031-33374-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ädvances in Knowledge Discovery and Data Mining"},
pages = {303–315},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Differentiable architecture search (DARTS) yields highly efficient gradient-based neural architecture search (NAS) by relaxing the discrete operation selection to optimize continuous architecture parameters that maps NAS from the discrete optimization to a continuous problem. DARTS then remaps the relaxed supernet back to the discrete space by one-off post-search pruning to obtain the final architecture (finalnet). Some emerging works argue that this remap is inherently prone to mismatch the network between training and evaluation which leads to performance discrepancy and even model collapse in extreme cases. We propose to close the gap between the relaxed supernet in training and the pruned finalnet in evaluation through utilizing small temperature to sparsify the continuous distribution in the training phase. To this end, we first formulate sparse-noisy softmax to get around gradient saturation. We then propose an exponential temperature schedule to better control the outbound distribution and elaborate an entropy-based adaptive scheme to finally achieve the enhancement. We conduct extensive experiments to verify the efficiency and efficacy of our method.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Differentiable architecture search (DARTS) yields highly efficient gradient-based neural architecture search (NAS) by relaxing the discrete operation selection to optimize continuous architecture parameters that maps NAS from the discrete optimization to a continuous problem. DARTS then remaps the relaxed supernet back to the discrete space by one-off post-search pruning to obtain the final architecture (finalnet). Some emerging works argue that this remap is inherently prone to mismatch the network between training and evaluation which leads to performance discrepancy and even model collapse in extreme cases. We propose to close the gap between the relaxed supernet in training and the pruned finalnet in evaluation through utilizing small temperature to sparsify the continuous distribution in the training phase. To this end, we first formulate sparse-noisy softmax to get around gradient saturation. We then propose an exponential temperature schedule to better control the outbound distribution and elaborate an entropy-based adaptive scheme to finally achieve the enhancement. We conduct extensive experiments to verify the efficiency and efficacy of our method.
2721.
Zhang, Peng; Zhang, Pengjv; Wu, Haoran; Tang, Jinsong
Automatic deep learning for automatic target detection of sonar images Proceedings Article
In: Ba, Shuhong; Zhou, Fan (Ed.): Third International Conference on Machine Learning and Computer Application (ICMLCA 2022), pp. 1263634, International Society for Optics and Photonics SPIE, 2023.
@inproceedings{10.1117/12.2675409,
title = {Automatic deep learning for automatic target detection of sonar images},
author = {Peng Zhang and Pengjv Zhang and Haoran Wu and Jinsong Tang},
editor = {Shuhong Ba and Fan Zhou},
url = {https://doi.org/10.1117/12.2675409},
doi = {10.1117/12.2675409},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Third International Conference on Machine Learning and Computer Application (ICMLCA 2022)},
volume = {12636},
pages = {1263634},
publisher = {SPIE},
organization = {International Society for Optics and Photonics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2720.
Huang, Jia-Cheng; Zeng, Guo-Qiang; Geng, Guang-Gang; Weng, Jian; Lu, Kang-Di; Zhang, Yu
In: Computers & Security, vol. 132, pp. 103310, 2023, ISSN: 0167-4048.
@article{HUANG2023103310,
title = {Differential evolution-based convolutional neural networks: An automatic architecture design method for intrusion detection in industrial control systems},
author = {Jia-Cheng Huang and Guo-Qiang Zeng and Guang-Gang Geng and Jian Weng and Kang-Di Lu and Yu Zhang},
url = {https://www.sciencedirect.com/science/article/pii/S0167404823002201},
doi = {https://doi.org/10.1016/j.cose.2023.103310},
issn = {0167-4048},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Computers & Security},
volume = {132},
pages = {103310},
abstract = {Industrial control systems (ICSs) are facing serious and evolving security threats because of a variety of malicious attacks. Deep learning-based intrusion detection systems (IDSs) have been widely considered as one of promising security solutions for ICSs, but these deep neural networks for IDSs in ICSs have been designed manually, which are extremely dependent on expert experience with numerous model parameters. This paper makes the first attempt to develop an automatic architecture design method of convolutional neural networks (CNNs) based on differential evolution (abbreviated as DE-CNN) for the intrusion detection issue in ICSs. The first phase of the proposed DE-CNN is the off-line architecture optimization of the CNNs constructed by three basic units such as ResNetBlockUnit, DenseNetBlockUnit, and PoolingUnit, including encoding the architecture parameters of a CNN as a population, evaluating the fitness of the population by the validation accuracy and the number of CNN model parameters, implementing the evolutionary process including mutation and crossover operations, and selecting the best individual from the population. Then, the optimal CNN model obtained by the off-line optimization of DE-CNN is deployed for the online IDSs. The experimental results on two intrusion detection datasets in ICSs including SWaT and WADI have demonstrated the superiority of the proposed DE-CNN to the state-of-the-art manually-designed and neuroevolution-based methods under both unsupervised and supervised learning in terms of Precision, Recall, F1-Score and the number of the CNN model parameters.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Industrial control systems (ICSs) are facing serious and evolving security threats because of a variety of malicious attacks. Deep learning-based intrusion detection systems (IDSs) have been widely considered as one of promising security solutions for ICSs, but these deep neural networks for IDSs in ICSs have been designed manually, which are extremely dependent on expert experience with numerous model parameters. This paper makes the first attempt to develop an automatic architecture design method of convolutional neural networks (CNNs) based on differential evolution (abbreviated as DE-CNN) for the intrusion detection issue in ICSs. The first phase of the proposed DE-CNN is the off-line architecture optimization of the CNNs constructed by three basic units such as ResNetBlockUnit, DenseNetBlockUnit, and PoolingUnit, including encoding the architecture parameters of a CNN as a population, evaluating the fitness of the population by the validation accuracy and the number of CNN model parameters, implementing the evolutionary process including mutation and crossover operations, and selecting the best individual from the population. Then, the optimal CNN model obtained by the off-line optimization of DE-CNN is deployed for the online IDSs. The experimental results on two intrusion detection datasets in ICSs including SWaT and WADI have demonstrated the superiority of the proposed DE-CNN to the state-of-the-art manually-designed and neuroevolution-based methods under both unsupervised and supervised learning in terms of Precision, Recall, F1-Score and the number of the CNN model parameters.
2719.
Wang, Zigeng; Li, Bingbing; Xiao, Xia; Zhang, Tianyun; Bragin, Mikhail A.; Yan, Bing; Ding, Caiwen; Rajasekaran, Sanguthevar
Automatic Subnetwork Search Through Dynamic Differentiable Neuron Pruning Proceedings Article
In: 2023 24th International Symposium on Quality Electronic Design (ISQED), pp. 1-6, 2023.
@inproceedings{10129379,
title = {Automatic Subnetwork Search Through Dynamic Differentiable Neuron Pruning},
author = {Zigeng Wang and Bingbing Li and Xia Xiao and Tianyun Zhang and Mikhail A. Bragin and Bing Yan and Caiwen Ding and Sanguthevar Rajasekaran},
url = {https://ieeexplore.ieee.org/abstract/document/10129379},
doi = {10.1109/ISQED57927.2023.10129379},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 24th International Symposium on Quality Electronic Design (ISQED)},
pages = {1-6},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}