Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2024
3118.
Tao, Devon; Bang, Lucas
Fitness Landscape Analysis of a Cell-Based Neural Architecture Search Space Conference
Proceedings of the 1st International Conference on Explainable AI for Neural and Symbolic Methods - Volume 1: EXPLAINS, INSTICC SciTePress, 2024, ISBN: 978-989-758-720-7.
@conference{explains24,
title = {Fitness Landscape Analysis of a Cell-Based Neural Architecture Search Space},
author = {Devon Tao and Lucas Bang},
doi = {10.5220/0012892400003886},
isbn = {978-989-758-720-7},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the 1st International Conference on Explainable AI for Neural and Symbolic Methods - Volume 1: EXPLAINS},
pages = {77-86},
publisher = {SciTePress},
organization = {INSTICC},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
3117.
Guo, Daidou; Qin, Chuan
PCMark-NAS: Lightweight Print-Camera Resilient Watermarking Networks via Neural Architecture Search Proceedings Article
In: Proceedings of the 6th ACM International Conference on Multimedia in Asia, Association for Computing Machinery, New York, NY, USA, 2024, ISBN: 9798400712739.
@inproceedings{10.1145/3696409.3700284,
title = {PCMark-NAS: Lightweight Print-Camera Resilient Watermarking Networks via Neural Architecture Search},
author = {Daidou Guo and Chuan Qin},
url = {https://doi.org/10.1145/3696409.3700284},
doi = {10.1145/3696409.3700284},
isbn = {9798400712739},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the 6th ACM International Conference on Multimedia in Asia},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
series = {MMAsia '24},
abstract = {Print-camera resilient image watermarking methods aim to ensure the integrity of watermark messages during print-camera channel transmission, thereby achieving copyright protection and traceability. Currently, to enhance the visual quality of watermarked images and improve robustness against print-camera noise, many deep learning-based print-camera resilient image watermarking networks are designed to be more complex, which increases the difficulty of deploying the model on mobile devices. Therefore, in this paper, we propose a method that utilizes Neural Architecture Search (NAS) to automatically learn a lightweight print-camera resilient image watermarking network structure, named PCMark-NAS. Experimental results show that, compared to traditional manual design watermarking networks, PCMark-NAS effectively reduces the parameters of the network while ensuring robustness and transparency through the formulation of appropriate search spaces and the design of corresponding search strategies and architectures.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Print-camera resilient image watermarking methods aim to ensure the integrity of watermark messages during print-camera channel transmission, thereby achieving copyright protection and traceability. Currently, to enhance the visual quality of watermarked images and improve robustness against print-camera noise, many deep learning-based print-camera resilient image watermarking networks are designed to be more complex, which increases the difficulty of deploying the model on mobile devices. Therefore, in this paper, we propose a method that utilizes Neural Architecture Search (NAS) to automatically learn a lightweight print-camera resilient image watermarking network structure, named PCMark-NAS. Experimental results show that, compared to traditional manual design watermarking networks, PCMark-NAS effectively reduces the parameters of the network while ensuring robustness and transparency through the formulation of appropriate search spaces and the design of corresponding search strategies and architectures.
3116.
Silva, Diego A.; Elsheikh, Ahmed; Smagulova, Kamilya; Fouda, Mohammed E.; Eltawil, Ahmed M.
Chimera: A Block-Based Neural Architecture Search Framework for Event-Based Object Detection Technical Report
2024.
@techreport{silva2024chimerablockbasedneuralarchitecture,
title = {Chimera: A Block-Based Neural Architecture Search Framework for Event-Based Object Detection},
author = {Diego A. Silva and Ahmed Elsheikh and Kamilya Smagulova and Mohammed E. Fouda and Ahmed M. Eltawil},
url = {https://arxiv.org/abs/2412.19646},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3115.
Ravindran, Saranya; Rajagopalan, Sasikumar
PNasFH-Net: Pyramid Neural Architecture Search Forward Network for Facial Emotion Recognition in Uncontrolled and Pose Variant Environment Journal Article
In: Knowledge-Based Systems, pp. 112944, 2024, ISSN: 0950-7051.
@article{RAVINDRAN2024112944,
title = {PNasFH-Net: Pyramid Neural Architecture Search Forward Network for Facial Emotion Recognition in Uncontrolled and Pose Variant Environment},
author = {Saranya Ravindran and Sasikumar Rajagopalan},
url = {https://www.sciencedirect.com/science/article/pii/S0950705124015788},
doi = {https://doi.org/10.1016/j.knosys.2024.112944},
issn = {0950-7051},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Knowledge-Based Systems},
pages = {112944},
abstract = {Face emotion recognition has attracted more attention in recent years because of its wide range of applications. This framework represents a novel facial emotion recognition technique known as a Pyramid Neural Architecture Search Forward Network (PNasFH-Net) developed for face emotion recognition. At first, the input image is subjected to image denoising and contrast enhancement using the Type II Fuzzy System and Cuckoo Search Optimization Algorithm (T2FCS) filter. Then, face detection is performed using the Viola-Jones algorithm based on the resultant denoised contrast-enhanced image. Subsequently, different features, like Spider Local Image Features (SLIF) with entropy, and Local Directional Number Pattern (LDNP) are extracted from the detected face in the feature extraction phase. Finally, facial emotions at different poses are recognized from the extracted features using PNasFH-Net. Here, PNasFH-Net is developed by the integration of the Deep Pyramidal Residual Network (PyramidNet) and NASNet. The recognized classes are surprise, sad, neutral, happy, fear, disgust, contempt and anger. The benchmark dataset for facial expression recognition, AffectNet is employed to assess the performance of the proposed model using performance measures, such as accuracy, TPR, and TNR. In addition, the developed PNasFH-Net obtained a higher accuracy of 90.315%, True Negative Rate (TNR) of 90.157% and True Positive Rate (TPR) of 91.047%.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Face emotion recognition has attracted more attention in recent years because of its wide range of applications. This framework represents a novel facial emotion recognition technique known as a Pyramid Neural Architecture Search Forward Network (PNasFH-Net) developed for face emotion recognition. At first, the input image is subjected to image denoising and contrast enhancement using the Type II Fuzzy System and Cuckoo Search Optimization Algorithm (T2FCS) filter. Then, face detection is performed using the Viola-Jones algorithm based on the resultant denoised contrast-enhanced image. Subsequently, different features, like Spider Local Image Features (SLIF) with entropy, and Local Directional Number Pattern (LDNP) are extracted from the detected face in the feature extraction phase. Finally, facial emotions at different poses are recognized from the extracted features using PNasFH-Net. Here, PNasFH-Net is developed by the integration of the Deep Pyramidal Residual Network (PyramidNet) and NASNet. The recognized classes are surprise, sad, neutral, happy, fear, disgust, contempt and anger. The benchmark dataset for facial expression recognition, AffectNet is employed to assess the performance of the proposed model using performance measures, such as accuracy, TPR, and TNR. In addition, the developed PNasFH-Net obtained a higher accuracy of 90.315%, True Negative Rate (TNR) of 90.157% and True Positive Rate (TPR) of 91.047%.
3114.
Yan, Xueming; Huang, Han; Jin, Yaochu; Wang, Zilong; Hao, Zhifeng
Neural Architecture Search Based on Bipartite Graphs for Text Classification Journal Article
In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2024.
@article{10817777,
title = {Neural Architecture Search Based on Bipartite Graphs for Text Classification},
author = {Xueming Yan and Han Huang and Yaochu Jin and Zilong Wang and Zhifeng Hao},
doi = {10.1109/TNNLS.2024.3514708},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
pages = {1-15},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3113.
Sarmiento-Rosales, Sergio; García, Jesús Llano; Falcón-Cardona, Jesús; Monroy, Raúl; Llano, Manuel Casillas; Sosa-Hernández, Víctor
Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: ECTA, INSTICC SciTePress, 2024, ISBN: 978-989-758-721-4.
@conference{ecta24,
title = {Surrogate Modeling for Efficient Evolutionary Multi-Objective Neural Architecture Search in Super Resolution Image Restoration},
author = {Sergio Sarmiento-Rosales and Jesús Llano García and Jesús Falcón-Cardona and Raúl Monroy and Manuel Casillas Llano and Víctor Sosa-Hernández},
doi = {10.5220/0012949000003837},
isbn = {978-989-758-721-4},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: ECTA},
pages = {242-249},
publisher = {SciTePress},
organization = {INSTICC},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
3112.
Yang, Zekang; Zeng, Wang; Jin, Sheng; Qian, Chen; Luo, Ping; Liu, Wentao
NADER: Neural Architecture Design via Multi-Agent Collaboration Technical Report
2024.
@techreport{yang2024naderneuralarchitecturedesign,
title = {NADER: Neural Architecture Design via Multi-Agent Collaboration},
author = {Zekang Yang and Wang Zeng and Sheng Jin and Chen Qian and Ping Luo and Wentao Liu},
url = {https://arxiv.org/abs/2412.19206},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3111.
Nguyen, Nam Hoang
Design and Optimization of Deep Neural Networks for Multi-Label Classification on Smart Electrical Devices Monitoring System Proceedings Article
In: 2024 13th International Conference on Control, Automation and Information Sciences (ICCAIS), pp. 1-5, 2024.
@inproceedings{10814375,
title = {Design and Optimization of Deep Neural Networks for Multi-Label Classification on Smart Electrical Devices Monitoring System},
author = {Nam Hoang Nguyen},
url = {https://ieeexplore.ieee.org/abstract/document/10814375},
doi = {10.1109/ICCAIS63750.2024.10814375},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 13th International Conference on Control, Automation and Information Sciences (ICCAIS)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3110.
Rajesh, Chilukamari; Sadam, Ravichandra; Kumar, Sushil
Automated Deep Learning Models for Medical Image Segmentation and Denoising Proceedings Article
In: 2024 17th International Conference on Signal Processing and Communication System (ICSPCS), pp. 1-7, 2024.
@inproceedings{10815837,
title = {Automated Deep Learning Models for Medical Image Segmentation and Denoising},
author = {Chilukamari Rajesh and Ravichandra Sadam and Sushil Kumar},
url = {https://ieeexplore.ieee.org/abstract/document/10815837},
doi = {10.1109/ICSPCS63175.2024.10815837},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 17th International Conference on Signal Processing and Communication System (ICSPCS)},
pages = {1-7},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3109.
Bansal, Divyansh; Al-Jawahry, Hassan M.; Al-Farouni, Mohammed; Kumar, Raman; Kaur, Kamaljeet; Bhosle, Nilesh
Neural Architecture Search (NAS) for Vision Tasks Proceedings Article
In: 2024 4th International Conference on Technological Advancements in Computational Sciences (ICTACS), pp. 1964-1969, 2024.
@inproceedings{10840421,
title = {Neural Architecture Search (NAS) for Vision Tasks},
author = {Divyansh Bansal and Hassan M. Al-Jawahry and Mohammed Al-Farouni and Raman Kumar and Kamaljeet Kaur and Nilesh Bhosle},
url = {https://ieeexplore.ieee.org/abstract/document/10840421},
doi = {10.1109/ICTACS62700.2024.10840421},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 4th International Conference on Technological Advancements in Computational Sciences (ICTACS)},
pages = {1964-1969},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3108.
Paolini, Emilio; Andriolli, Nicola; Cococcioni, Marco
Enhancing Neuromorphic Photonic Hardware Performance through Neural Architecture Search Proceedings Article
In: 2024.
@inproceedings{inproceedingsd,
title = {Enhancing Neuromorphic Photonic Hardware Performance through Neural Architecture Search},
author = {Emilio Paolini and Nicola Andriolli and Marco Cococcioni},
url = {https://www.researchgate.net/publication/384014716_Enhancing_Neuromorphic_Photonic_Hardware_Performance_through_Neural_Architecture_Search},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3107.
Wang, Zhenrong; Li, Weifeng; Xie, Jinghao; Li, Bin
An Automatic Design Method for Dynamic Detection Networks of Industrial Surface Defects Proceedings Article
In: 2024 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), pp. 1449-1453, 2024.
@inproceedings{10857221,
title = {An Automatic Design Method for Dynamic Detection Networks of Industrial Surface Defects},
author = {Zhenrong Wang and Weifeng Li and Jinghao Xie and Bin Li},
url = {https://ieeexplore.ieee.org/abstract/document/10857221},
doi = {10.1109/IEEM62345.2024.10857221},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)},
pages = {1449-1453},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3106.
(Ed.)
Multi-objective evolutionary algorithm based graph neural network architecture search Collection
2024.
@collection{nokey,
title = {Multi-objective evolutionary algorithm based graph neural network architecture search},
author = {
Lianyi He and Xiaobo Liu and Hongbo Xiang and Guangjun Wang
},
url = {https://isciia-itca.bit.edu.cn/docs/2024-11/8a328d718bf341c38f497fa293107827.pdf},
year = {2024},
date = {2024-01-01},
booktitle = {ISCIIA-ITCA 2024},
journal = {ISCIIA-ITCA 2024},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3105.
(Ed.)
NAS-FL:Fingerprint Localization Method Based on Automatically Designed Neural Network Architecture Collection
2024.
@collection{liu-ipin24a,
title = {NAS-FL:Fingerprint Localization Method Based on Automatically Designed Neural Network Architecture},
author = {Wen Liu and Haoyue Jiang and Ran Li and Zhongliang Deng},
url = {https://ceur-ws.org/Vol-3919/short1.pdf},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = { WiP Proceedings of the Fourteenth International Conference on Indoor Positioning and Indoor Navigation - Work-in-Progress Papers (IPIN-WiP 2024)},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3104.
Lan, Zheng; Wang, Bo; Liu, Senting; Shi, Xiupeng
CoDANet: Hardware-Aware Neural Architecture Search for Optimized Deep Learning on FPGAs Proceedings Article
In: 2024 2nd International Conference on Artificial Intelligence and Automation Control (AIAC), pp. 563-568, 2024.
@inproceedings{10899512,
title = {CoDANet: Hardware-Aware Neural Architecture Search for Optimized Deep Learning on FPGAs},
author = {Zheng Lan and Bo Wang and Senting Liu and Xiupeng Shi},
url = {https://ieeexplore.ieee.org/abstract/document/10899512},
doi = {10.1109/AIAC63745.2024.10899512},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {2024 2nd International Conference on Artificial Intelligence and Automation Control (AIAC)},
pages = {563-568},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3103.
Ayebo, Iyanu; Olatunde, Emma
Neural Architecture Search: Automating Deep Learning Model Design Journal Article
In: 2024.
@article{articleh,
title = {Neural Architecture Search: Automating Deep Learning Model Design},
author = {Iyanu Ayebo and Emma Olatunde},
url = {https://www.researchgate.net/publication/390131112_Neural_Architecture_Search_Automating_Deep_Learning_Model_Design},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3102.
Krstic, Lazar; Ivanovic, Milos; Simic, Visnja; Stojanovic, Boban
Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks Journal Article
In: Egyptian Informatics Journal, vol. 28, pp. 100581, 2024, ISSN: 1110-8665.
@article{KRSTIC2024100581b,
title = {Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks},
author = {Lazar Krstic and Milos Ivanovic and Visnja Simic and Boban Stojanovic},
url = {https://www.sciencedirect.com/science/article/pii/S1110866524001440},
doi = {https://doi.org/10.1016/j.eij.2024.100581},
issn = {1110-8665},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Egyptian Informatics Journal},
volume = {28},
pages = {100581},
abstract = {The paper presents the GeNNsem (Genetic algorithm ANNs ensemble) software framework for the simultaneous optimization of individual neural networks and building their optimal ensemble. The proposed framework employs a genetic algorithm to search for suitable architectures and hyperparameters of the individual neural networks to maximize the weighted sum of accuracy and diversity in their predictions. The optimal ensemble consists of networks with low errors but diverse predictions, resulting in a more generalized model. The scalability of the proposed framework is ensured by utilizing micro-services and Kubernetes batching orchestration. GeNNsem has been evaluated on two regression benchmark problems and compared with related machine learning techniques. The proposed approach exhibited supremacy over other ensemble approaches and individual neural networks in all common regression modeling metrics. Real-world use-case experiments in the domain of hydro-informatics have further demonstrated the main advantages of GeNNsem: requires the least training sessions for individual models when optimizing an ensemble; networks in an ensemble are generally simple due to the regularization provided by a trivial initial population and custom genetic operators; execution times are reduced by two orders of magnitude as a result of parallelization.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The paper presents the GeNNsem (Genetic algorithm ANNs ensemble) software framework for the simultaneous optimization of individual neural networks and building their optimal ensemble. The proposed framework employs a genetic algorithm to search for suitable architectures and hyperparameters of the individual neural networks to maximize the weighted sum of accuracy and diversity in their predictions. The optimal ensemble consists of networks with low errors but diverse predictions, resulting in a more generalized model. The scalability of the proposed framework is ensured by utilizing micro-services and Kubernetes batching orchestration. GeNNsem has been evaluated on two regression benchmark problems and compared with related machine learning techniques. The proposed approach exhibited supremacy over other ensemble approaches and individual neural networks in all common regression modeling metrics. Real-world use-case experiments in the domain of hydro-informatics have further demonstrated the main advantages of GeNNsem: requires the least training sessions for individual models when optimizing an ensemble; networks in an ensemble are generally simple due to the regularization provided by a trivial initial population and custom genetic operators; execution times are reduced by two orders of magnitude as a result of parallelization.
2023
3101.
(Ed.)
Designing a New Search Space for Multivariate Time-Series Neural Architecture Search Collection
2023.
@collection{MacKinnon-ecmlw23a,
title = {Designing a New Search Space for Multivariate Time-Series Neural Architecture Search},
author = {Christopher MacKinnon and Robert Atkinson},
url = {https://books.google.de/books?hl=de&lr=&id=iFHqEAAAQBAJ&oi=fnd&pg=PA190&dq=%22neural+architecture+search%22&ots=CTD1RKtbFl&sig=vM1TNI0RduFF3A62Q04s-Ls7ZbM#v=onepage&q=%22neural%20architecture%20search%22&f=false},
year = {2023},
date = {2023-12-23},
urldate = {2023-12-23},
booktitle = {Advanced Analytics and Learning on Temporal Data: 8th ECML PKDD Workshop ...},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3100.
Zhao, Junbo; Ning, Xuefei; Liu, Enshu; Ru, Binxin; Zhou, Zixuan; Zhao, Tianchen; Chen, Chen; Zhang, Jiajin; Liao, Qingmin; Wang, Yu
Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS “Cold-Start” Technical Report
2023.
@techreport{Zhao22,
title = {Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS “Cold-Start”},
author = {Junbo Zhao and Xuefei Ning and Enshu Liu and Binxin Ru and Zixuan Zhou and Tianchen Zhao and Chen Chen and Jiajin Zhang and Qingmin Liao and Yu Wang},
url = {https://nicsefc.ee.tsinghua.edu.cn/nics_file/pdf/4208e529-772e-4977-be31-0b7cc4c7a9fc.pdf},
year = {2023},
date = {2023-12-20},
urldate = {2023-12-20},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3099.
Zhang, Baochang; Xu, Sheng; Lin, Mingbao; Wang, Tiancheng; Doermann, David
Binary Neural Networks: Algorithms, Architectures, and Applications Book
2023.
@book{ZhangBNN23a,
title = {Binary Neural Networks: Algorithms, Architectures, and Applications },
author = {Baochang Zhang and Sheng Xu and Mingbao Lin and Tiancheng Wang and David Doermann},
url = {https://www.taylorfrancis.com/books/mono/10.1201/9781003376132/binary-neural-networks-baochang-zhang-sheng-xu-mingbao-lin-tiancheng-wang-david-doermann},
year = {2023},
date = {2023-12-13},
keywords = {},
pubstate = {published},
tppubtype = {book}
}
3098.
(Ed.)
Multi-task Graph Neural Architecture Search with Task-aware Collaboration and Curriculum Collection
2023.
@collection{Qin-neurips23a,
title = {Multi-task Graph Neural Architecture Search with Task-aware Collaboration and Curriculum},
author = {Yijian Qin and Xin Wang and Ziwei Zhang and Hong Chen and Wenwu Zhu},
url = {https://openreview.net/pdf?id=TOxpAwp0VE},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {37th Conference on Neural Information Processing Systems (NeurIPS 2023).},
journal = {37th Conference on Neural Information Processing Systems (NeurIPS 2023)},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3097.
(Ed.)
Unsupervised Graph Neural Architecture Search with Disentangled Self-supervision Collection
2023.
@collection{Zhang-neurips23a,
title = {Unsupervised Graph Neural Architecture Search with Disentangled Self-supervision},
author = {Zeyang Zhang and Xin Wang and Ziwei Zhang and Guangyao Shen and Shiqi Shen and Wenwu Zhu},
url = {https://openreview.net/pdf?id=UAFa5ZhR85},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {37th Conference on Neural Information Processing Systems (NeurIPS 2023).},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3096.
(Ed.)
MathNAS: If Blocks Have a Role in Mathematical Architecture Design Collection
2023.
@collection{Wang-neurips23a,
title = {MathNAS: If Blocks Have a Role in Mathematical Architecture Design},
author = {Qinsi Wang and Jinghan Ke and Zhi Liang and Sihai Zhang},
url = {https://openreview.net/pdf?id=e1l4ZYprQH},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {37th Conference on Neural Information Processing Systems (NeurIPS 2023)},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3095.
(Ed.)
KD-Zero: Evolving Knowledge Distiller for Any Teacher-Student Pairs Collection
2023.
@collection{Li-neurips23a,
title = {KD-Zero: Evolving Knowledge Distiller for Any Teacher-Student Pairs},
author = {Lujun Li and Peijie Dong and Anggeng Li and Zimian Wei and Ya Yang},
url = {https://openreview.net/pdf?id=OlMKa5YZ8e},
year = {2023},
date = {2023-12-01},
booktitle = {37th Conference on Neural Information Processing Systems (NeurIPS 2023)},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3094.
(Ed.)
Analyzing Generalization of Neural Networks through Loss Path Kernels Collection
2023.
@collection{Chen-neurips23a,
title = {Analyzing Generalization of Neural Networks through Loss Path Kernels},
author = {Yilan Chen and Wei Huang and Hao Wang and Charlotte Loh and Akash Srivastava and Lam M.Nguyen and Tsui-Wei Weng},
url = {https://openreview.net/pdf?id=8Ba7VJ7xiM},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {37th Conference on Neural Information Processing Systems (NeurIPS 2023)},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3093.
(Ed.)
Operation-Level Early Stopping for Robustifying Differentiable NAS Collection
2023.
@collection{Jiang-neurips23a,
title = {Operation-Level Early Stopping for Robustifying Differentiable NAS},
author = {Shen Jiang and Zipeng Ji and Guanghui Zhu and Chunfeng Yuan},
url = {https://openreview.net/pdf?id=yAOwkf4FyL},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {37th Conference on Neural Information Processing Systems (NeurIPS 2023).},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3092.
(Ed.)
AutoGO: Automated Computation Graph Optimization for Neural Network Evolution Collection
2023.
@collection{Salameh-neurips23a,
title = {AutoGO: Automated Computation Graph Optimization for Neural Network Evolution},
author = {Mohammad Salameh and Keith G. Mills and Negar Hassanpour and Fred X. Han and Shuting Zhang and Wei Lu and Shangling Jui and Chunhua Zhou and Fengyu Sun and Di Niu},
url = {https://openreview.net/pdf?id=lDI3ZuyzM9},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {37th Conference on Neural Information Processing Systems (NeurIPS 2023)},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3091.
(Ed.)
Building A Mobile Text Recognizer via Truncated SVD-based Knowledge Distillation-Guided NAS Collection
2023.
@collection{Lin-bmvc2023,
title = {Building A Mobile Text Recognizer via Truncated SVD-based Knowledge Distillation-Guided NAS},
author = {Weifeng Lin and Canyu Xie and Dezhi Peng and Jiapeng Wang and Lianwen Jin and Wei Ding and Cong Yao and Mengchao He},
url = {https://papers.bmvc2023.org/0375.pdf},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {The 34th British Machine Vision Conference},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3090.
Wang, Shang; Tang, Huangrong
GRADIENT-FREE PROXY FOR EFFICIENT LANGUAGE MODEL SEARCH Technical Report
2023.
@techreport{Wang-rg23a,
title = {GRADIENT-FREE PROXY FOR EFFICIENT LANGUAGE MODEL SEARCH},
author = {Shang Wang and Huangrong Tang},
url = {https://www.researchgate.net/profile/Shang-Wang-23/publication/376072656_GRADIENT-FREE_PROXY_FOR_EFFICIENT_LANGUAGE_MODEL_SEARCH/links/6568af28b1398a779dc7962b/GRADIENT-FREE-PROXY-FOR-EFFICIENT-LANGUAGE-MODEL-SEARCH.pdf},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3089.
Hundhausen, Felix; Hubschneider, Simon; Asfou, Tamim
Grasping with Humanoid Hands based on In-Hand Vision and Hardware-accelerated CNNs Technical Report
2023.
@techreport{Hundhausen-23a,
title = {Grasping with Humanoid Hands based on In-Hand Vision and Hardware-accelerated CNNs},
author = {Felix Hundhausen and Simon Hubschneider and Tamim Asfou},
url = {https://h2t.iar.kit.edu/pdf/Hundhausen2023.pdf},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3088.
Xue, Yu; Tong, Weinan; Neri, Ferrante; Chen, Peng; Luo, Tao; Zhen, Liangli; Wang, Xiao
Evolutionary Architecture Search for Generative Adversarial Networks Based On Weight Sharing Technical Report
2023.
@techreport{Xue-23a,
title = {Evolutionary Architecture Search for Generative Adversarial Networks Based On Weight Sharing},
author = {Yu Xue and Weinan Tong and Ferrante Neri and Peng Chen and Tao Luo and Liangli Zhen and Xiao Wang},
url = {https://s3.eu-central-1.amazonaws.com/eu-st01.ext.exlibrisgroup.com/44SUR_INST/storage/alma/6C/5F/BC/AE/64/94/51/5E/79/33/7C/E9/F1/86/8F/A6/TEVC_00119_2023_manuscript_R2.pdf?response-content-type=application%2Fpdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231220T093633Z&X-Amz-SignedHeaders=host&X-Amz-Expires=119&X-Amz-Credential=AKIAJN6NPMNGJALPPWAQ%2F20231220%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Signature=ef6f0f810a9a364744dfb3260a0578af25980e9662f1017ace22a03fe7be693b},
year = {2023},
date = {2023-12-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3087.
Al-Sabri, Raeed; Gao, Jianliang; Chen, Jiamin; Oloulade, Babatounde Moctard; Lyu, Tengfei
AutoTGRL: an automatic text-graph representation learning framework Journal Article
In: Neural Computing and Applications, 2023.
@article{Sabri-nca23a,
title = {AutoTGRL: an automatic text-graph representation learning framework},
author = {
Raeed Al-Sabri and Jianliang Gao and Jiamin Chen and Babatounde Moctard Oloulade and Tengfei Lyu
},
url = {https://link.springer.com/article/10.1007/s00521-023-09226-0},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
journal = { Neural Computing and Applications},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3086.
SASNAUSKAS, PAULIUS; PETKEVIČIUS, INAS
Symbolic Neural Architecture Search for Differential Equations Journal Article
In: 2023.
@article{SASNAUSKAS-ieeeaccess23a,
title = {Symbolic Neural Architecture Search for Differential Equations},
author = {PAULIUS SASNAUSKAS and INAS PETKEVIČIUS},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10354328&tag=1},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3085.
Luo, Xiangzhong
Hardware-aware neural architecture search and compression towards embedded intelligence PhD Thesis
2023.
@phdthesis{LuoPHD23a,
title = {Hardware-aware neural architecture search and compression towards embedded intelligence},
author = {Luo, Xiangzhong},
url = {https://dr.ntu.edu.sg/handle/10356/172506},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3084.
Huai, Shuo
Enabling efficient edge intelligence: a hardware-software codesign approach PhD Thesis
2023.
@phdthesis{Huai-phd23a,
title = {Enabling efficient edge intelligence: a hardware-software codesign approach},
author = {Huai, Shuo},
url = {https://dr.ntu.edu.sg/handle/10356/172499},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3083.
(Ed.)
Improving Natural Language Understanding with Computation-Efficient Retrieval Representation Fusion Collection
2023.
@collection{nokey,
title = {Improving Natural Language Understanding with Computation-Efficient Retrieval Representation Fusion},
author = {Shangyu Wu and Ying Xiong and Yufei Cui and Xue Liu and Buzhou Tang and Tei-WEi Kuo and Chun Jason Xue},
url = {https://neurips2023-enlsp.github.io/papers/paper_79.pdf},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {The third version of the Efficient Natural Language and Speech Processing (ENLSP-III) workshop, NeurIPS2023},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3082.
Park, Soohyun; Son, Seok Bin; Lee, Youn Kyu; Jung, Soyi; Kim, Joongheon
Two-stage architectural fine-tuning for neural architecture search in efficient transfer learning Journal Article
In: ELECTRONICS LETTERS , vol. 59, no. 24, 2023.
@article{Park-el23a,
title = {Two-stage architectural fine-tuning for neural architecture search in efficient transfer learning},
author = {Soohyun Park and Seok Bin Son and Youn Kyu Lee and Soyi Jung and Joongheon Kim},
url = {https://ietresearch.onlinelibrary.wiley.com/doi/pdfdirect/10.1049/ell2.13066},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
journal = {ELECTRONICS LETTERS },
volume = {59},
number = {24},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3081.
Saluky, Saluky; Nugraha, Gusti Baskara; Supangkat, Suhono Harso
Enhancing Abandoned Object Detection with Dual Background Models and Yolo-NAS Journal Article
In: International Journal of Intelligent Systems and Applications in Engineering, vol. 12, no. 2, pp. 547–554, 2023.
@article{Saluky_Nugraha_Supangkat_2023,
title = {Enhancing Abandoned Object Detection with Dual Background Models and Yolo-NAS},
author = {Saluky Saluky and Gusti Baskara Nugraha and Suhono Harso Supangkat},
url = {https://ijisae.org/index.php/IJISAE/article/view/4298},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
journal = {International Journal of Intelligent Systems and Applications in Engineering},
volume = {12},
number = {2},
pages = {547–554},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3080.
Bellodi, E.; Bertozzi, D.; Bizzarri, A.; Favalli, M.; Fraccaroli, M.; Zese, R.
Efficient Resource-Aware Neural Architecture Search with a Neuro-Symbolic Approach Proceedings Article
In: 2023 IEEE 16th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC), pp. 171-178, IEEE Computer Society, Los Alamitos, CA, USA, 2023.
@inproceedings{10387830,
title = {Efficient Resource-Aware Neural Architecture Search with a Neuro-Symbolic Approach},
author = {E. Bellodi and D. Bertozzi and A. Bizzarri and M. Favalli and M. Fraccaroli and R. Zese},
url = {https://doi.ieeecomputersociety.org/10.1109/MCSoC60832.2023.00034},
doi = {10.1109/MCSoC60832.2023.00034},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {2023 IEEE 16th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC)},
pages = {171-178},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Hardware-aware Neural Architectural Search (NAS) is gaining momentum to enable the deployment of deep learning on edge devices with limited computing capabilities. Incorporating device-related objectives such as affordable floating point operations, latency, power, memory usage, etc. into the optimization process makes searching for the most efficient neural architecture more complicated, since both model accuracy and hardware cost should guide the search. The main concern with most state-of-the-art hardware-aware NAS strategies is that they propose for evaluation also trivially infeasible network models for the capabilities of the hardware platform at hand. Moreover, previously generated models are frequently not exploited to intelligently generate new ones, leading to prohibitive computational costs for practical relevance. This paper aims to boost the computational efficiency of hardware-aware NAS by means of a neuro-symbolic framework revolving around a Probabilistic Inductive Logic Programming module to define and exploit a set of symbolic rules. This component learns and refines the probabilities associated with the rules, allowing the framework to adapt and improve over time, thus quickly narrowing down the search space toward the most promising neural architectures.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Hardware-aware Neural Architectural Search (NAS) is gaining momentum to enable the deployment of deep learning on edge devices with limited computing capabilities. Incorporating device-related objectives such as affordable floating point operations, latency, power, memory usage, etc. into the optimization process makes searching for the most efficient neural architecture more complicated, since both model accuracy and hardware cost should guide the search. The main concern with most state-of-the-art hardware-aware NAS strategies is that they propose for evaluation also trivially infeasible network models for the capabilities of the hardware platform at hand. Moreover, previously generated models are frequently not exploited to intelligently generate new ones, leading to prohibitive computational costs for practical relevance. This paper aims to boost the computational efficiency of hardware-aware NAS by means of a neuro-symbolic framework revolving around a Probabilistic Inductive Logic Programming module to define and exploit a set of symbolic rules. This component learns and refines the probabilities associated with the rules, allowing the framework to adapt and improve over time, thus quickly narrowing down the search space toward the most promising neural architectures.
3079.
Kapoor, A.; Soans, R.; Dixit, S.; Ns, P.; Singh, B.; Das, M.
NASEREX: Optimizing Early Exits via AutoML for Scalable Efficient Inference in Big Image Streams Proceedings Article
In: 2023 IEEE International Conference on Big Data (BigData), pp. 5266-5271, IEEE Computer Society, Los Alamitos, CA, USA, 2023.
@inproceedings{10386502,
title = {NASEREX: Optimizing Early Exits via AutoML for Scalable Efficient Inference in Big Image Streams},
author = {A. Kapoor and R. Soans and S. Dixit and P. Ns and B. Singh and M. Das},
url = {https://doi.ieeecomputersociety.org/10.1109/BigData59044.2023.10386502},
doi = {10.1109/BigData59044.2023.10386502},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {2023 IEEE International Conference on Big Data (BigData)},
pages = {5266-5271},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {We investigate the problem of smart operational efficiency, at scale, in Machine Learning models for Big Data streams, in context of embedded AI applications, by learning optimal early exits. Embedded AI applications that employ deep neural models depend on efficient model inference at scale, especially on resource-constrained hardware. Recent vision/text/audio models are computationally complex with huge parameter spaces and input samples typically pass through multiple layers, each with large tensor computations, to produce valid outputs. Generally, in most real scenarios, AI applications deal with big data streams, such as streams of audio signals, static images and/or high resolution video frames. Deep ML models powering such applications have to continuously perform inference on such big data streams for varied tasks such as noise suppression, face detection, gait estimation and so on. Ensuring efficiency is challenging, even with model compression techniques since they reduce model size but often fail to achieve scalable inference efficiency over continuous streams. Early exits enable adaptive inference by extracting valid outputs from any pre-final layer of a deep model which significantly boosts efficiency at scale since many of the input instances need not be processed at all the layers of a deep model, especially for big streams. Suitable early exit structure design (number + positions) is a difficult but crucial aspect in improving efficiency without any loss in predictive performance, especially in context of big streams. Naive manual early exit design that does not consider the hardware capacity or data stream characteristics is counterproductive. We propose NASEREX framework that leverages Neural architecture Search (NAS) with a novel saliency-constrained search space and exit decision metric to learn suitable early exit structure to augment Deep Neural models for scalable efficient inference on big image streams. Optimized exit-augmented models perform $approx 2.5 times$ faster having $approx 4 times$ aggregated lower effective FLOPs, with no significant accuracy loss.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We investigate the problem of smart operational efficiency, at scale, in Machine Learning models for Big Data streams, in context of embedded AI applications, by learning optimal early exits. Embedded AI applications that employ deep neural models depend on efficient model inference at scale, especially on resource-constrained hardware. Recent vision/text/audio models are computationally complex with huge parameter spaces and input samples typically pass through multiple layers, each with large tensor computations, to produce valid outputs. Generally, in most real scenarios, AI applications deal with big data streams, such as streams of audio signals, static images and/or high resolution video frames. Deep ML models powering such applications have to continuously perform inference on such big data streams for varied tasks such as noise suppression, face detection, gait estimation and so on. Ensuring efficiency is challenging, even with model compression techniques since they reduce model size but often fail to achieve scalable inference efficiency over continuous streams. Early exits enable adaptive inference by extracting valid outputs from any pre-final layer of a deep model which significantly boosts efficiency at scale since many of the input instances need not be processed at all the layers of a deep model, especially for big streams. Suitable early exit structure design (number + positions) is a difficult but crucial aspect in improving efficiency without any loss in predictive performance, especially in context of big streams. Naive manual early exit design that does not consider the hardware capacity or data stream characteristics is counterproductive. We propose NASEREX framework that leverages Neural architecture Search (NAS) with a novel saliency-constrained search space and exit decision metric to learn suitable early exit structure to augment Deep Neural models for scalable efficient inference on big image streams. Optimized exit-augmented models perform $approx 2.5 times$ faster having $approx 4 times$ aggregated lower effective FLOPs, with no significant accuracy loss.
3078.
Heuillet, Alexandre
Exploring deep neural network differentiable architecture design PhD Thesis
Université Paris-Saclay, 2023.
@phdthesis{heuillet:tel-04420933,
title = {Exploring deep neural network differentiable architecture design},
author = {Alexandre Heuillet},
url = {https://hal.science/tel-04420933},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
number = {2023UPASG069},
school = {Université Paris-Saclay},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3077.
Chen, Hui; Li, Nannan; Chen, Rong
Ni-DehazeNet: representation learning via bilevel optimized architecture search for nighttime dehazing Journal Article
In: The Visual Computer, 2023.
@article{Chen-vc23a,
title = {Ni-DehazeNet: representation learning via bilevel optimized architecture search for nighttime dehazing},
author = {
Hui Chen and Nannan Li and Rong Chen
},
url = {https://link.springer.com/article/10.1007/s00371-023-03159-4},
year = {2023},
date = {2023-11-28},
journal = {The Visual Computer},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3076.
Lyu, Zonglei; Yu, Tong; Pan, Fuxi; Zhang, Yilin; Luo, Jia; Zhang, Dan; Chen, Yiren; Zhang, Bo; Li, Guangyao
A survey of model compression strategies for object detection Journal Article
In: Multimedia Tools and Applications , 2023.
@article{nokey,
title = {A survey of model compression strategies for object detection},
author = {
Zonglei Lyu and Tong Yu and Fuxi Pan and Yilin Zhang and Jia Luo and Dan Zhang and Yiren Chen and Bo Zhang and Guangyao Li
},
url = {https://link.springer.com/article/10.1007/s11042-023-17192-x},
year = {2023},
date = {2023-11-02},
urldate = {2023-11-02},
journal = {Multimedia Tools and Applications },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3075.
POYSER, MATTHEW
2023.
@phdthesis{POYSER-phd2023,
title = {Minimizing Computational Resources for Deep Machine Learning: A Compression and Neural Architecture Search Perspective for Image Classification and Object Detection},
author = {POYSER, MATTHEW},
url = {http://etheses.dur.ac.uk/15207/1/main.pdf},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3074.
Liu, Shiya
Energy-efficient Neuromorphic Computing for Resource-constrained Internet of Things Devices PhD Thesis
2023.
@phdthesis{Liu-phd23a,
title = {Energy-efficient Neuromorphic Computing for Resource-constrained Internet of Things Devices},
author = {Liu, Shiya},
url = {https://vtechworks.lib.vt.edu/handle/10919/116629},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3073.
Xie, Tao; Zhang, Haoming; Yang, Linqi; Wang, Ke; Dai, Kun; Li, Ruifeng; Zhao, Lijun
Point-NAS: A Novel Neural Architecture Search Framework for Point Cloud Analysis Journal Article
In: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, vol. PP, 2023, ISSN: 1057-7149.
@article{PMID:37963007,
title = {Point-NAS: A Novel Neural Architecture Search Framework for Point Cloud Analysis},
author = {Tao Xie and Haoming Zhang and Linqi Yang and Ke Wang and Kun Dai and Ruifeng Li and Lijun Zhao},
url = {https://doi.org/10.1109/TIP.2023.3331223},
doi = {10.1109/tip.2023.3331223},
issn = {1057-7149},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
journal = {IEEE transactions on image processing : a publication of the IEEE Signal Processing Society},
volume = {PP},
abstract = {Recently, point-based networks have exhibited extraordinary potential for 3D point cloud processing. However, owing to the meticulous design of both parameters and hyperparameters inside the network, constructing a promising network for each point cloud task can be an expensive endeavor. In this work, we develop a novel one-shot search framework called Point-NAS to automatically determine optimum architectures for various point cloud tasks. Specifically, we design an elastic feature extraction (EFE) module that serves as a basic unit for architecture search, which expands seamlessly alongside both the width and depth of the network for efficient feature extraction. Based on the EFE module, we devise a searching space, which is encoded into a supernet to provide a wide number of latent network structures for a particular point cloud task. To fully optimize the weights of the supernet, we propose a weight coupling sandwich rule that samples the largest, smallest, and multiple medium models at each iteration and fuses their gradients to update the supernet. Furthermore, we present a united gradient adjustment algorithm that mitigates gradient conflict induced by distinct gradient directions of sampled models and supernet, thus expediting the convergence of the supernet and assuring that it can be comprehensively trained. Pursuant to the provided techniques, the trained supernet enables a multitude of subnets to be incredibly well-optimized. Finally, we conduct an evolutionary search for the supernet under resource constraints to find promising architectures for different tasks. Experimentally, the searched Point-NAS with weights inherited from the supernet realizes outstanding results across a variety of benchmarks. i.e., 94.2% and 88.9% overall accuracy under ModelNet40 and ScanObjectNN, 68.6% mIoU under S3DIS, 63.6% and 69.3% mAP@0.25 under SUN RGB-D and ScanNet V2 datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recently, point-based networks have exhibited extraordinary potential for 3D point cloud processing. However, owing to the meticulous design of both parameters and hyperparameters inside the network, constructing a promising network for each point cloud task can be an expensive endeavor. In this work, we develop a novel one-shot search framework called Point-NAS to automatically determine optimum architectures for various point cloud tasks. Specifically, we design an elastic feature extraction (EFE) module that serves as a basic unit for architecture search, which expands seamlessly alongside both the width and depth of the network for efficient feature extraction. Based on the EFE module, we devise a searching space, which is encoded into a supernet to provide a wide number of latent network structures for a particular point cloud task. To fully optimize the weights of the supernet, we propose a weight coupling sandwich rule that samples the largest, smallest, and multiple medium models at each iteration and fuses their gradients to update the supernet. Furthermore, we present a united gradient adjustment algorithm that mitigates gradient conflict induced by distinct gradient directions of sampled models and supernet, thus expediting the convergence of the supernet and assuring that it can be comprehensively trained. Pursuant to the provided techniques, the trained supernet enables a multitude of subnets to be incredibly well-optimized. Finally, we conduct an evolutionary search for the supernet under resource constraints to find promising architectures for different tasks. Experimentally, the searched Point-NAS with weights inherited from the supernet realizes outstanding results across a variety of benchmarks. i.e., 94.2% and 88.9% overall accuracy under ModelNet40 and ScanObjectNN, 68.6% mIoU under S3DIS, 63.6% and 69.3% mAP@0.25 under SUN RGB-D and ScanNet V2 datasets.
3072.
Dimanov, Daniel
2023.
@phdthesis{Dimanov-phd23a,
title = {Efficient Multi-Objective NeuroEvolution in Computer Vision and Applications for Threat Identification},
author = {Daniel Dimanov},
url = {http://eprints.bournemouth.ac.uk/39138/1/DIMANOV%2C%20Daniel_Ph.D._2023.pdf},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3071.
Li, Yueyang; Wu, Zhouejie; Shen, Junfei; Zhang, Qican
In: Opt. Express, vol. 31, no. 24, pp. 40803–40823, 2023.
@article{Li:23,
title = {Real-time 3D shape measurement of dynamic scenes using fringe projection profilometry: lightweight NAS-optimized dual frequency deep learning approach},
author = {Yueyang Li and Zhouejie Wu and Junfei Shen and Qican Zhang},
url = {https://opg.optica.org/oe/abstract.cfm?URI=oe-31-24-40803},
doi = {10.1364/OE.506343},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
journal = {Opt. Express},
volume = {31},
number = {24},
pages = {40803–40823},
publisher = {Optica Publishing Group},
abstract = {Achieving real-time and high-accuracy 3D reconstruction of dynamic scenes is a fundamental challenge in many fields, including online monitoring, augmented reality, and so on. On one hand, traditional methods, such as Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP), are struggling to balance measuring efficiency and accuracy. On the other hand, deep learning-based approaches, which offer the potential for improved accuracy, are hindered by large parameter amounts and complex structures less amenable to real-time requirements. To solve this problem, we proposed a network architecture search (NAS)-based method for real-time processing and 3D measurement of dynamic scenes with rate equivalent to single-shot. A NAS-optimized lightweight neural network was designed for efficient phase demodulation, while an improved dual-frequency strategy was employed coordinately for flexible absolute phase unwrapping. The experiment results demonstrate that our method can effectively perform 3D reconstruction with a reconstruction speed of 58fps, and realize high-accuracy measurement of dynamic scenes based on deep learning for what we believe to be the first time with the average RMS error of about 0.08 mm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Achieving real-time and high-accuracy 3D reconstruction of dynamic scenes is a fundamental challenge in many fields, including online monitoring, augmented reality, and so on. On one hand, traditional methods, such as Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP), are struggling to balance measuring efficiency and accuracy. On the other hand, deep learning-based approaches, which offer the potential for improved accuracy, are hindered by large parameter amounts and complex structures less amenable to real-time requirements. To solve this problem, we proposed a network architecture search (NAS)-based method for real-time processing and 3D measurement of dynamic scenes with rate equivalent to single-shot. A NAS-optimized lightweight neural network was designed for efficient phase demodulation, while an improved dual-frequency strategy was employed coordinately for flexible absolute phase unwrapping. The experiment results demonstrate that our method can effectively perform 3D reconstruction with a reconstruction speed of 58fps, and realize high-accuracy measurement of dynamic scenes based on deep learning for what we believe to be the first time with the average RMS error of about 0.08 mm.
3070.
Xie, G.; Li, Q.; Shi, Z.; Fang, H.; Ji, S.; Jiang, Y.; Yuan, Z.; Ma, L.; Xu, M.
Generating Neural Networks for Diverse Networking Classification Tasks via Hardware-Aware Neural Architecture Search Journal Article
In: IEEE Transactions on Computers, no. 01, pp. 1-14, 2023, ISSN: 1557-9956.
@article{10323250,
title = {Generating Neural Networks for Diverse Networking Classification Tasks via Hardware-Aware Neural Architecture Search},
author = {G. Xie and Q. Li and Z. Shi and H. Fang and S. Ji and Y. Jiang and Z. Yuan and L. Ma and M. Xu},
url = {https://www.computer.org/csdl/journal/tc/5555/01/10323250/1SewSv3Y4VO},
doi = {10.1109/TC.2023.3333253},
issn = {1557-9956},
year = {2023},
date = {2023-11-01},
urldate = {5555-11-01},
journal = {IEEE Transactions on Computers},
number = {01},
pages = {1-14},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Neural networks (NNs) are widely used in classification-based networking analysis to help traffic transmission and system security. However, there are heterogeneous network devices (e.g., switches and routers) in a network. Manually customizing NNs with specific device requirements (e.g., max allowed running latency) can be time-consuming and labor-intensive. Furthermore, the diverse data characteristics of different networking classification tasks add to the burden of NN customization. This paper introduces Loong, a neural architecture search (NAS) based system that automatically generates NNs for various networking tasks and devices. Loong includes a neural operation embedding module, which embeds candidate neural operations into the layer to be designed. Then, the layer-wise training is used to generate a task-specific NN layer by layer. This layer-wise scheme simultaneously trains and selects candidate neural operations using gradient feedback. Finally, only the important operations are selected to form the layer, maximizing accuracy. By incorporating multiple objectives, including deployment memory and running latency of devices, into the training and selection of NNs, Loong is able to customize NNs for heterogeneous network devices. Experiments show that Loong’s NNs outperform 13 manual-designed and NAS-based NNs, with a 4.11% improvement in F1-score. Additionally, Loong’s NNs achieve faster (7.92X) speeds on commodity devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural networks (NNs) are widely used in classification-based networking analysis to help traffic transmission and system security. However, there are heterogeneous network devices (e.g., switches and routers) in a network. Manually customizing NNs with specific device requirements (e.g., max allowed running latency) can be time-consuming and labor-intensive. Furthermore, the diverse data characteristics of different networking classification tasks add to the burden of NN customization. This paper introduces Loong, a neural architecture search (NAS) based system that automatically generates NNs for various networking tasks and devices. Loong includes a neural operation embedding module, which embeds candidate neural operations into the layer to be designed. Then, the layer-wise training is used to generate a task-specific NN layer by layer. This layer-wise scheme simultaneously trains and selects candidate neural operations using gradient feedback. Finally, only the important operations are selected to form the layer, maximizing accuracy. By incorporating multiple objectives, including deployment memory and running latency of devices, into the training and selection of NNs, Loong is able to customize NNs for heterogeneous network devices. Experiments show that Loong’s NNs outperform 13 manual-designed and NAS-based NNs, with a 4.11% improvement in F1-score. Additionally, Loong’s NNs achieve faster (7.92X) speeds on commodity devices.
3069.
Wang, G.; Li, C.; Yuan, L.; Peng, J.; Xian, X.; Liang, X.; Chang, X.; Lin, L.
DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions Journal Article
In: IEEE Transactions on Pattern Analysis & Machine Intelligence, no. 01, pp. 1-18, 2023, ISSN: 1939-3539.
@article{10324326,
title = {DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions},
author = {G. Wang and C. Li and L. Yuan and J. Peng and X. Xian and X. Liang and X. Chang and L. Lin},
url = {https://www.computer.org/csdl/journal/tp/5555/01/10324326/1SgbFgBOI7u},
doi = {10.1109/TPAMI.2023.3335261},
issn = {1939-3539},
year = {2023},
date = {2023-11-01},
urldate = {5555-11-01},
journal = {IEEE Transactions on Pattern Analysis & Machine Intelligence},
number = {01},
pages = {1-18},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Neural Architecture Search (NAS), aiming at automatically designing neural architectures by machines, has been considered a key step toward automatic machine learning. One notable NAS branch is the weight-sharing NAS, which significantly improves search efficiency and allows NAS algorithms to run on ordinary computers. Despite receiving high expectations, this category of methods suffers from low search effectiveness. By employing a generalization boundedness tool, we demonstrate that the devil behind this drawback is the untrustworthy architecture rating with the oversized search space of the possible architectures. Addressing this problem, we modularize a large search space into blocks with small search spaces and develop a family of models with the distilling neural architecture (DNA) techniques. These proposed models, namely a DNA family, are capable of resolving multiple dilemmas of the weight-sharing NAS, such as scalability, efficiency, and multi-modal compatibility. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using heuristic algorithms. Moreover, under a certain computational complexity constraint, our method can seek architectures with different depths and widths. Extensive experimental evaluations show that our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively. Additionally, we provide in-depth empirical analysis and insights into neural architecture ratings. Codes available: https://github.com/changlin31/DNA.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Architecture Search (NAS), aiming at automatically designing neural architectures by machines, has been considered a key step toward automatic machine learning. One notable NAS branch is the weight-sharing NAS, which significantly improves search efficiency and allows NAS algorithms to run on ordinary computers. Despite receiving high expectations, this category of methods suffers from low search effectiveness. By employing a generalization boundedness tool, we demonstrate that the devil behind this drawback is the untrustworthy architecture rating with the oversized search space of the possible architectures. Addressing this problem, we modularize a large search space into blocks with small search spaces and develop a family of models with the distilling neural architecture (DNA) techniques. These proposed models, namely a DNA family, are capable of resolving multiple dilemmas of the weight-sharing NAS, such as scalability, efficiency, and multi-modal compatibility. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using heuristic algorithms. Moreover, under a certain computational complexity constraint, our method can seek architectures with different depths and widths. Extensive experimental evaluations show that our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively. Additionally, we provide in-depth empirical analysis and insights into neural architecture ratings. Codes available: https://github.com/changlin31/DNA.