Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2022
1915.
Yu, Hongjiu; Sun, Qiancheng; Hu, Jin; Xue, Xingyuan; Luo, Jixiang; He, Dailan; Li, Yilong; Wang, Pengbo; Wang, Yuanyuan; Dai, Yaxu; Wang, Yan; Qin, Hongwei
Evaluating the Practicality of Learned Image Compression Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2207-14524,
title = {Evaluating the Practicality of Learned Image Compression},
author = {Hongjiu Yu and Qiancheng Sun and Jin Hu and Xingyuan Xue and Jixiang Luo and Dailan He and Yilong Li and Pengbo Wang and Yuanyuan Wang and Yaxu Dai and Yan Wang and Hongwei Qin},
url = {https://doi.org/10.48550/arXiv.2207.14524},
doi = {10.48550/arXiv.2207.14524},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2207.14524},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1914.
Ma, Jiaxiang
Pruning threshold search algorithm combined with PDARTS Proceedings Article
In: Wang, Lidan; Cen, Mengyi (Milly) (Ed.): 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022), pp. 382 – 387, International Society for Optics and Photonics SPIE, 2022.
@inproceedings{10.1117/12.2640465,
title = {Pruning threshold search algorithm combined with PDARTS},
author = {Jiaxiang Ma},
editor = {Lidan Wang and Mengyi (Milly) Cen},
url = {https://doi.org/10.1117/12.2640465},
doi = {10.1117/12.2640465},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022)},
volume = {12257},
pages = {382 -- 387},
publisher = {SPIE},
organization = {International Society for Optics and Photonics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1913.
Liu, Yang; Lu, Jun
Double Loss Block Neural Architecture Search Proceedings Article
In: 2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), pp. 731-734, 2022.
@inproceedings{9836540,
title = {Double Loss Block Neural Architecture Search},
author = {Yang Liu and Jun Lu},
url = {https://ieeexplore.ieee.org/abstract/document/9836540},
doi = {10.1109/ITAIC54216.2022.9836540},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC)},
volume = {10},
pages = {731-734},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1912.
Mariama, Diallo; Sun, Liang
i-DARTS: Improving differentiable architecture search by using graph and few-shot learning Proceedings Article
In: 2022 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), pp. 14-19, 2022.
@inproceedings{9844464,
title = {i-DARTS: Improving differentiable architecture search by using graph and few-shot learning},
author = {Diallo Mariama and Liang Sun},
url = {https://ieeexplore.ieee.org/abstract/document/9844464},
doi = {10.1109/ICAICA54878.2022.9844464},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA)},
pages = {14-19},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1911.
Pau, Danilo; Ambrose, Prem Kumar
Automated Neural and On-Device Learning for Micro Controllers Proceedings Article
In: 2022 IEEE 21st Mediterranean Electrotechnical Conference (MELECON), pp. 758-763, 2022.
@inproceedings{9843050,
title = {Automated Neural and On-Device Learning for Micro Controllers},
author = {Danilo Pau and Prem Kumar Ambrose},
url = {https://ieeexplore.ieee.org/abstract/document/9843050},
doi = {10.1109/MELECON53508.2022.9843050},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 IEEE 21st Mediterranean Electrotechnical Conference (MELECON)},
pages = {758-763},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1910.
Pang, Tianji; Zhao, Shijie; Han, Junwei; Zhang, Shu; Guo, Lei; Liu, Tianming
Gumbel-Softmax based Neural Architecture Search for Hierarchical Brain Networks Decomposition Journal Article
In: Medical Image Analysis, pp. 102570, 2022, ISSN: 1361-8415.
@article{PANG2022102570,
title = {Gumbel-Softmax based Neural Architecture Search for Hierarchical Brain Networks Decomposition},
author = {Tianji Pang and Shijie Zhao and Junwei Han and Shu Zhang and Lei Guo and Tianming Liu},
url = {https://www.sciencedirect.com/science/article/pii/S1361841522002110},
doi = {https://doi.org/10.1016/j.media.2022.102570},
issn = {1361-8415},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Medical Image Analysis},
pages = {102570},
abstract = {Understanding the brain's functional architecture has been an important topic in the neuroimaging field. A variety of brain network modeling methods have been proposed. Recently, deep neural network-based methods have shown a great advantage in modeling the hierarchical and complex functional brain networks (FBNs). However, most of these deep neural networks were handcrafted, making it time-consuming to find the relatively optimal architecture. To address this problem, we propose a novel unsupervised differentiable neural architecture search (NAS) algorithm, named Gumbel-Softmax based Neural Architecture Search (GS-NAS), to automate the architecture design of deep belief network (DBN) for hierarchical FBN decomposition. Specifically, we introduce the Gumbel-Softmax scheme to reframe the discrete architecture sampling procedure during NAS to be continuous. Guided by the reconstruction error minimization procedure, the architecture search can be driven by the intrinsic functional architecture of the brain, thereby revealing the possible hierarchical functional brain organization via DBN structure. The proposed GS-NAS algorithm can simultaneously optimize the number of hidden units for each layer and the network depth. Extensive experiment results on both task and resting-state functional magnetic resonance imaging data have demonstrated the effectiveness and efficiency of the proposed GS-NAS model. The identified hierarchically organized FBNs provide novel insight into understanding human brain function.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Understanding the brain's functional architecture has been an important topic in the neuroimaging field. A variety of brain network modeling methods have been proposed. Recently, deep neural network-based methods have shown a great advantage in modeling the hierarchical and complex functional brain networks (FBNs). However, most of these deep neural networks were handcrafted, making it time-consuming to find the relatively optimal architecture. To address this problem, we propose a novel unsupervised differentiable neural architecture search (NAS) algorithm, named Gumbel-Softmax based Neural Architecture Search (GS-NAS), to automate the architecture design of deep belief network (DBN) for hierarchical FBN decomposition. Specifically, we introduce the Gumbel-Softmax scheme to reframe the discrete architecture sampling procedure during NAS to be continuous. Guided by the reconstruction error minimization procedure, the architecture search can be driven by the intrinsic functional architecture of the brain, thereby revealing the possible hierarchical functional brain organization via DBN structure. The proposed GS-NAS algorithm can simultaneously optimize the number of hidden units for each layer and the network depth. Extensive experiment results on both task and resting-state functional magnetic resonance imaging data have demonstrated the effectiveness and efficiency of the proposed GS-NAS model. The identified hierarchically organized FBNs provide novel insight into understanding human brain function.
1909.
Tong, Lyuyang; Du, Bo
Neural architecture search via reference point based multi-objective evolutionary algorithm Journal Article
In: Pattern Recognition, pp. 108962, 2022, ISSN: 0031-3203.
@article{TONG2022108962,
title = {Neural architecture search via reference point based multi-objective evolutionary algorithm},
author = {Lyuyang Tong and Bo Du},
url = {https://www.sciencedirect.com/science/article/pii/S0031320322004423},
doi = {https://doi.org/10.1016/j.patcog.2022.108962},
issn = {0031-3203},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Pattern Recognition},
pages = {108962},
abstract = {For neural architecture search, NSGA-Net has searched a representative neural architecture set of Pareto-optimal solutions to consider both accuracy and computation complexity simultaneously. However, some decision-makers only concentrate on such neural architectures in the subpart regions of Pareto-optimal Frontier that they have interests in. Under the above circumstances, certain uninterested neural architectures may cost many computing resources. In order to consider the preference of decision-makers, we propose the reference point based NSGA-Net (RNSGA-Net) for neural architecture search. The core of RNSGA-Net adopts the reference point approach to guarantee the Pareto-optimal region close to the reference points and also combines the advantage of NSGAII with the fast nondominated sorting approach to split the Pareto front. Moreover, we augment an extra bit value of the original encoding to represent two types of residual block and one type of dense block for residual connection and dense connection in the RNSGA-Net. In order to satisfy the decision-maker preference, the multi-objective is measured to search competitive neural architecture by minimizing an error metric and FLOPs of computational complexity. Experiment results on the CIFAR-10 dataset demonstrate that RNSGA-Net can improve NSGA-Net in terms of the more structured representation space and the preference of decision-makers.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
For neural architecture search, NSGA-Net has searched a representative neural architecture set of Pareto-optimal solutions to consider both accuracy and computation complexity simultaneously. However, some decision-makers only concentrate on such neural architectures in the subpart regions of Pareto-optimal Frontier that they have interests in. Under the above circumstances, certain uninterested neural architectures may cost many computing resources. In order to consider the preference of decision-makers, we propose the reference point based NSGA-Net (RNSGA-Net) for neural architecture search. The core of RNSGA-Net adopts the reference point approach to guarantee the Pareto-optimal region close to the reference points and also combines the advantage of NSGAII with the fast nondominated sorting approach to split the Pareto front. Moreover, we augment an extra bit value of the original encoding to represent two types of residual block and one type of dense block for residual connection and dense connection in the RNSGA-Net. In order to satisfy the decision-maker preference, the multi-objective is measured to search competitive neural architecture by minimizing an error metric and FLOPs of computational complexity. Experiment results on the CIFAR-10 dataset demonstrate that RNSGA-Net can improve NSGA-Net in terms of the more structured representation space and the preference of decision-makers.
1908.
Lu, Zhichao; Cheng, Ran; Jin, Yaochu; Tan, Kay Chen; Deb, Kalyanmoy
Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment Technical Report
2022.
@techreport{lu2022neural,
title = {Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment},
author = {Zhichao Lu and Ran Cheng and Yaochu Jin and Kay Chen Tan and Kalyanmoy Deb},
url = {https://arxiv.org/abs/2208.04321},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {arXiv preprint arXiv:2208.04321},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1907.
Zhang, Shi-Xin; Hsieh, Chang-Yu; Zhang, Shengyu; Yao, Hong
Differentiable quantum architecture search Journal Article
In: Quantum Science and Technology, 2022.
@article{10.1088/2058-9565/ac87cd,
title = {Differentiable quantum architecture search},
author = {Shi-Xin Zhang and Chang-Yu Hsieh and Shengyu Zhang and Hong Yao},
url = {http://iopscience.iop.org/article/10.1088/2058-9565/ac87cd},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Quantum Science and Technology},
abstract = {Quantum architecture search (QAS) is the process of automating architecture engineering of quantum circuits. It has been desired to construct a powerful and general QAS platform which can significantly accelerate current efforts to identify quantum advantages of error-prone and depth-limited quantum circuits in the NISQ era. Hereby, we propose a general framework of differentiable quantum architecture search (DQAS), which enables automated designs of quantum circuits in an end-to-end differentiable fashion. We present several examples of circuit design problems to demonstrate the power of DQAS. For instance, unitary operations are decomposed into quantum gates, noisy circuits are re-designed to improve accuracy, and circuit layouts for quantum approximation optimization algorithm are automatically discovered and upgraded for combinatorial optimization problems. These results not only manifest the vast potential of DQAS being an essential tool for the NISQ application developments, but also present an interesting research topic from the theoretical perspective as it draws inspirations from the newly emerging interdisciplinary paradigms of differentiable programming, probabilistic programming, and quantum programming.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Quantum architecture search (QAS) is the process of automating architecture engineering of quantum circuits. It has been desired to construct a powerful and general QAS platform which can significantly accelerate current efforts to identify quantum advantages of error-prone and depth-limited quantum circuits in the NISQ era. Hereby, we propose a general framework of differentiable quantum architecture search (DQAS), which enables automated designs of quantum circuits in an end-to-end differentiable fashion. We present several examples of circuit design problems to demonstrate the power of DQAS. For instance, unitary operations are decomposed into quantum gates, noisy circuits are re-designed to improve accuracy, and circuit layouts for quantum approximation optimization algorithm are automatically discovered and upgraded for combinatorial optimization problems. These results not only manifest the vast potential of DQAS being an essential tool for the NISQ application developments, but also present an interesting research topic from the theoretical perspective as it draws inspirations from the newly emerging interdisciplinary paradigms of differentiable programming, probabilistic programming, and quantum programming.
1906.
Peng, Jie; Liu, Haijun; Zhao, Zhongjin; Li, Zhiwei; Liu, Sen; Li, Qingjiang
CMQ: Crossbar-aware Neural Network Mixed-precision Quantization via Differentiable Architecture Search Journal Article
In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2022.
@article{9852786,
title = {CMQ: Crossbar-aware Neural Network Mixed-precision Quantization via Differentiable Architecture Search},
author = {Jie Peng and Haijun Liu and Zhongjin Zhao and Zhiwei Li and Sen Liu and Qingjiang Li},
doi = {10.1109/TCAD.2022.3197495},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1905.
Xu, Lumin; Jin, Sheng; Liu, Wentao; Qian, Chen; Ouyang, Wanli; Luo, Ping; Wang, Xiaogang
ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild Journal Article
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-18, 2022.
@article{9852279,
title = {ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild},
author = {Lumin Xu and Sheng Jin and Wentao Liu and Chen Qian and Wanli Ouyang and Ping Luo and Xiaogang Wang},
url = {https://ieeexplore.ieee.org/abstract/document/9852279},
doi = {10.1109/TPAMI.2022.3197352},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
pages = {1-18},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1904.
Ye, Peng; Li, Baopu; Chen, Tao; Fan, Jiayuan; Mei, Zhen; Lin, Chen; Zuo, Chongyan; Chi, Qinghua; Ouyan, Wanli
Efficient Joint-Dimensional Search with Solution Space Regularization for Real-Time Semantic Segmentation Miscellaneous
2022.
@misc{https://doi.org/10.48550/arxiv.2208.05271,
title = {Efficient Joint-Dimensional Search with Solution Space Regularization for Real-Time Semantic Segmentation},
author = {Peng Ye and Baopu Li and Tao Chen and Jiayuan Fan and Zhen Mei and Chen Lin and Chongyan Zuo and Qinghua Chi and Wanli Ouyan},
url = {https://arxiv.org/abs/2208.05271},
doi = {10.48550/ARXIV.2208.05271},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
publisher = {arXiv},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
1903.
Wang, Zhen; Wei, Zhewei; Li, Yaliang; Kuang, Weirui; Ding, Bolin
Graph Neural Networks with Node-Wise Architecture Proceedings Article
In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1949–1958, Association for Computing Machinery, Washington DC, USA, 2022, ISBN: 9781450393850.
@inproceedings{10.1145/3534678.3539387,
title = {Graph Neural Networks with Node-Wise Architecture},
author = {Zhen Wang and Zhewei Wei and Yaliang Li and Weirui Kuang and Bolin Ding},
url = {https://doi.org/10.1145/3534678.3539387},
doi = {10.1145/3534678.3539387},
isbn = {9781450393850},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
pages = {1949–1958},
publisher = {Association for Computing Machinery},
address = {Washington DC, USA},
series = {KDD '22},
abstract = {Recently, Neural Architecture Search (NAS) for GNN has received increasing popularity as it can seek an optimal architecture for a given new graph. However, the optimal architecture is applied to all the instances (i.e., nodes, in the context of graph) equally, which might be insufficient to handle the diverse local patterns ingrained in a graph, as shown in this paper and some very recent studies. Thus, we argue the necessity of node-wise architecture search for GNN. Nevertheless, node-wise architecture cannot be realized by trivially applying NAS methods node by node due to the scalability issue and the need for determining test nodes' architectures. To tackle these challenges, we propose a framework wherein the parametric controllers decide the GNN architecture for each node based on its local patterns. We instantiate our framework with depth, aggregator and resolution controllers, and then elaborate on learning the backbone GNN model and the controllers to encourage their cooperation. Empirically, we justify the effects of node-wise architecture through the performance improvements introduced by the three controllers, respectively. Moreover, our proposed framework significantly outperforms state-of-the-art methods on five of the ten real-world datasets, where the diversity of these datasets has hindered any graph convolution-based method to lead on them simultaneously. This result further confirms that node-wise architecture can help GNNs become versatile models.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Recently, Neural Architecture Search (NAS) for GNN has received increasing popularity as it can seek an optimal architecture for a given new graph. However, the optimal architecture is applied to all the instances (i.e., nodes, in the context of graph) equally, which might be insufficient to handle the diverse local patterns ingrained in a graph, as shown in this paper and some very recent studies. Thus, we argue the necessity of node-wise architecture search for GNN. Nevertheless, node-wise architecture cannot be realized by trivially applying NAS methods node by node due to the scalability issue and the need for determining test nodes' architectures. To tackle these challenges, we propose a framework wherein the parametric controllers decide the GNN architecture for each node based on its local patterns. We instantiate our framework with depth, aggregator and resolution controllers, and then elaborate on learning the backbone GNN model and the controllers to encourage their cooperation. Empirically, we justify the effects of node-wise architecture through the performance improvements introduced by the three controllers, respectively. Moreover, our proposed framework significantly outperforms state-of-the-art methods on five of the ten real-world datasets, where the diversity of these datasets has hindered any graph convolution-based method to lead on them simultaneously. This result further confirms that node-wise architecture can help GNNs become versatile models.
1902.
Lu, Zhichao; Cheng, Ran; Huang, Shihua; Zhang, Haoming; Qiu, Changxiao; Yang, Fan
Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation Technical Report
2022.
@techreport{lu2022surrogate,
title = {Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation},
author = {Zhichao Lu and Ran Cheng and Shihua Huang and Haoming Zhang and Changxiao Qiu and Fan Yang},
url = {https://arxiv.org/abs/2208.06820},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {arXiv preprint arXiv:2208.06820},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1901.
Lopes, Vasco; Santos, Miguel; Degardin, Bruno; Alexandre, Lu'is A
Guided Evolutionary Neural Architecture Search With Efficient Performance Estimation Technical Report
2022.
@techreport{lopes2022guided,
title = {Guided Evolutionary Neural Architecture Search With Efficient Performance Estimation},
author = {Vasco Lopes and Miguel Santos and Bruno Degardin and Lu'is A Alexandre},
url = {https://arxiv.org/abs/2208.06475},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {arXiv preprint arXiv:2208.06475},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1900.
Öchoa, Gabriela; Veerapen, Nadarajen"
Neural Architecture Search: A Visual Analysis Proceedings Article
In: Rudolph, Günter; Kononova, Anna V.; Aguirre, Hernán; Kerschke, Pascal; Ochoa, Gabriela; Tušar, Tea (Ed.): Parallel Problem Solving from Nature -- PPSN XVII, pp. 603–615, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-14714-2.
@inproceedings{10.1007/978-3-031-14714-2_42,
title = {Neural Architecture Search: A Visual Analysis},
author = {Gabriela Öchoa and Nadarajen" Veerapen},
editor = {Günter Rudolph and Anna V. Kononova and Hernán Aguirre and Pascal Kerschke and Gabriela Ochoa and Tea Tušar},
url = {https://link.springer.com/chapter/10.1007/978-3-031-14714-2_42},
isbn = {978-3-031-14714-2},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Parallel Problem Solving from Nature -- PPSN XVII},
pages = {603--615},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Neural architecture search (NAS) refers to the use of search heuristics to optimise the topology of deep neural networks. NAS algorithms have produced topologies that outperform human-designed ones. However, contrasting alternative NAS methods is difficult. To address this, several tabular NAS benchmarks have been proposed that exhaustively evaluate all architectures in a given search space. We conduct a thorough fitness landscape analysis of a popular tabular, cell-based NAS benchmark. Our results indicate that NAS landscapes are multi-modal, but have a relatively low number of local optima, from which it is not hard to escape. We confirm that reducing the noise in estimating performance reduces the number of local optima. We hypothesise that local-search based NAS methods are likely to be competitive, which we confirm by implementing a landscape-aware iterated local search algorithm that can outperform more elaborate evolutionary and reinforcement learning NAS methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural architecture search (NAS) refers to the use of search heuristics to optimise the topology of deep neural networks. NAS algorithms have produced topologies that outperform human-designed ones. However, contrasting alternative NAS methods is difficult. To address this, several tabular NAS benchmarks have been proposed that exhaustively evaluate all architectures in a given search space. We conduct a thorough fitness landscape analysis of a popular tabular, cell-based NAS benchmark. Our results indicate that NAS landscapes are multi-modal, but have a relatively low number of local optima, from which it is not hard to escape. We confirm that reducing the noise in estimating performance reduces the number of local optima. We hypothesise that local-search based NAS methods are likely to be competitive, which we confirm by implementing a landscape-aware iterated local search algorithm that can outperform more elaborate evolutionary and reinforcement learning NAS methods.
1899.
Chen, Yaofo; Guo, Yong; Chen, Peihao; Wang, Jingdong; Wang, Yaowei; Song, Hengjie; Tan, Mingkui
Automatic Subspace Evoking for Efficient Neural Architecture Search Technical Report
2022.
@techreport{chen2022automatic,
title = {Automatic Subspace Evoking for Efficient Neural Architecture Search},
author = {Yaofo Chen and Yong Guo and Peihao Chen and Jingdong Wang and Yaowei Wang and Hengjie Song and Mingkui Tan},
url = {https://chenyaofo.com/papers/chen-automatic-subspace-evoking-for-efficient-neural-architecture-search.pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {arXiv preprint},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1898.
Eslami, Saeedeh; Monsefi, Reza; Akbari, Mohammad
Towards Leveraging Structure for Neural Predictor in NAS Journal Article
In: Computer and Knowledge Engineering, pp. -, 2022, ISSN: 2538-5453.
@article{nokey,
title = {Towards Leveraging Structure for Neural Predictor in NAS},
author = {Saeedeh Eslami and Reza Monsefi and Mohammad Akbari},
url = {https://cke.um.ac.ir/article_42708.html},
doi = {10.22067/cke.2022.73356.1031},
issn = {2538-5453},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Computer and Knowledge Engineering},
pages = {-},
publisher = {Ferdowsi University of Mashhad},
abstract = {Neural Architecture Search (NAS), which automatically designs a neural architecture for a specific task, has attracted much attention in recent years. Properly defining the search space is a key step in the success of NAS approaches, which allows us to reduce the required time for evaluation. Thus, late strategies for searching a NAS space is to leverage supervised learning models for ranking potential neural models, i.e., surrogate predictive models. The predictive model takes the specification of an architecture (or its feature representation) and predicts the probable efficiency of the model ahead of training. Therefore, proper representation of a candidate architecture is an important factor for a predictor NAS approach. While several works have been devoted to training a good surrogate model, there exits limited research focusing on learning a good representation for these neural models. To address this problem, we investigate how to learn a representation with both structural and non-structural features of a network. In particular, we propose a tree structured encoding which permits to fully represent both networks’ layers and their intra-connections. The encoding is easily extendable to larger or more complex structures. Extensive experiments on two NAS datasets, NasBench101 and NasBench201, demonstrate the effectiveness of the proposed method as compared with the state-of-the-art predictors.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Architecture Search (NAS), which automatically designs a neural architecture for a specific task, has attracted much attention in recent years. Properly defining the search space is a key step in the success of NAS approaches, which allows us to reduce the required time for evaluation. Thus, late strategies for searching a NAS space is to leverage supervised learning models for ranking potential neural models, i.e., surrogate predictive models. The predictive model takes the specification of an architecture (or its feature representation) and predicts the probable efficiency of the model ahead of training. Therefore, proper representation of a candidate architecture is an important factor for a predictor NAS approach. While several works have been devoted to training a good surrogate model, there exits limited research focusing on learning a good representation for these neural models. To address this problem, we investigate how to learn a representation with both structural and non-structural features of a network. In particular, we propose a tree structured encoding which permits to fully represent both networks’ layers and their intra-connections. The encoding is easily extendable to larger or more complex structures. Extensive experiments on two NAS datasets, NasBench101 and NasBench201, demonstrate the effectiveness of the proposed method as compared with the state-of-the-art predictors.
1897.
Zhou, Tong; Ren, Shaolei; Xu, Xiaolin
ObfuNAS: A Neural Architecture Search-based DNN Obfuscation Approach Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-08569,
title = {ObfuNAS: A Neural Architecture Search-based DNN Obfuscation Approach},
author = {Tong Zhou and Shaolei Ren and Xiaolin Xu},
url = {https://doi.org/10.48550/arXiv.2208.08569},
doi = {10.48550/arXiv.2208.08569},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.08569},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1896.
Xiao, Tesi; Xiao, Xia; Chen, Ming; Chen, Youlong
Field-wise Embedding Size Search via Structural Hard Auxiliary Mask Pruning for Click-Through Rate Prediction Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-08004,
title = {Field-wise Embedding Size Search via Structural Hard Auxiliary Mask Pruning for Click-Through Rate Prediction},
author = {Tesi Xiao and Xia Xiao and Ming Chen and Youlong Chen},
url = {https://doi.org/10.48550/arXiv.2208.08004},
doi = {10.48550/arXiv.2208.08004},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.08004},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1895.
Bhardwaj, Kartikeya; Ward, James; Tung, Caleb; Gope, Dibakar; Meng, Lingchuan; Fedorov, Igor; Chalfin, Alex; Whatmough, Paul N.; Loh, Danny
Restructurable Activation Networks Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-08562,
title = {Restructurable Activation Networks},
author = {Kartikeya Bhardwaj and James Ward and Caleb Tung and Dibakar Gope and Lingchuan Meng and Igor Fedorov and Alex Chalfin and Paul N. Whatmough and Danny Loh},
url = {https://doi.org/10.48550/arXiv.2208.08562},
doi = {10.48550/arXiv.2208.08562},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.08562},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1894.
Zhang, Xuanyang; Li, Yonggang; Zhang, Xiangyu; Wang, Yongtao; Sun, Jian
Differentiable Architecture Search with Random Features Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-08835,
title = {Differentiable Architecture Search with Random Features},
author = {Xuanyang Zhang and Yonggang Li and Xiangyu Zhang and Yongtao Wang and Jian Sun},
url = {https://doi.org/10.48550/arXiv.2208.08835},
doi = {10.48550/arXiv.2208.08835},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.08835},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1893.
Chen, Hanxiong; Li, Yunqi; Zhu, He; Zhang, Yongfeng
Learn Basic Skills and Reuse: Modularized Adaptive Neural Architecture Search (MANAS) Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-11083,
title = {Learn Basic Skills and Reuse: Modularized Adaptive Neural Architecture Search (MANAS)},
author = {Hanxiong Chen and Yunqi Li and He Zhu and Yongfeng Zhang},
url = {https://doi.org/10.48550/arXiv.2208.11083},
doi = {10.48550/arXiv.2208.11083},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.11083},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1892.
Yu, Zhewen; Bouganis, Christos-Savvas
SVD-NAS: Coupling Low-Rank Approximation and Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-10404,
title = {SVD-NAS: Coupling Low-Rank Approximation and Neural Architecture Search},
author = {Zhewen Yu and Christos-Savvas Bouganis},
url = {https://doi.org/10.48550/arXiv.2208.10404},
doi = {10.48550/arXiv.2208.10404},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.10404},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1891.
Liu, Jing; Cai, Jianfei; Zhuang, Bohan
FocusFormer: Focusing on What We Need via Architecture Sampler Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-10861,
title = {FocusFormer: Focusing on What We Need via Architecture Sampler},
author = {Jing Liu and Jianfei Cai and Bohan Zhuang},
url = {https://doi.org/10.48550/arXiv.2208.10861},
doi = {10.48550/arXiv.2208.10861},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.10861},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1890.
Cao, Xuyang; Chen, Houjin; Li, Yanfeng; Peng, Yahui; Zhou, Yue; Cheng, Lin; Liu, Tianming; Shen, Dinggang
Auto-DenseUNet: Searchable neural network architecture for mass segmentation in 3D automated breast ultrasound Journal Article
In: Medical Image Analysis, pp. 102589, 2022, ISSN: 1361-8415.
@article{CAO2022102589,
title = {Auto-DenseUNet: Searchable neural network architecture for mass segmentation in 3D automated breast ultrasound},
author = {Xuyang Cao and Houjin Chen and Yanfeng Li and Yahui Peng and Yue Zhou and Lin Cheng and Tianming Liu and Dinggang Shen},
url = {https://www.sciencedirect.com/science/article/pii/S1361841522002250},
doi = {https://doi.org/10.1016/j.media.2022.102589},
issn = {1361-8415},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Medical Image Analysis},
pages = {102589},
abstract = {Accurate segmentation of breast mass in 3D automated breast ultrasound (ABUS) plays an important role in breast cancer analysis. Deep convolutional networks have become a promising approach in segmenting ABUS images. However, designing an effective network architecture is time-consuming, and highly relies on specialist’s experience and prior knowledge. To address this issue, we introduce a searchable segmentation network (denoted as Auto-DenseUNet) based on the neural architecture search (NAS) to search the optimal architecture automatically for the ABUS mass segmentation task. Concretely, a novel search space is designed based on a densely connected structure to enhance the gradient and information flows throughout the network. Then, to encourage multiscale information fusion, a set of searchable multiscale aggregation nodes between the down-sampling and up-sampling parts of the network are further designed. Thus, all the operators within the dense connection structure or between any two aggregation nodes can be searched to find the optimal structure. Finally, a novel decoupled search training strategy during architecture search is also introduced to alleviate the memory limitation caused by continuous relaxation in NAS. The proposed Auto-DeseUNet method has been evaluated on our ABUS dataset with 170 volumes (from 107 patients), including 120 training volumes and 50 testing volumes split at patient level. Experimental results on testing volumes show that our searched architecture performed better than several human-designed segmentation models on the 3D ABUS mass segmentation task, indicating the effectiveness of our proposed method.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Accurate segmentation of breast mass in 3D automated breast ultrasound (ABUS) plays an important role in breast cancer analysis. Deep convolutional networks have become a promising approach in segmenting ABUS images. However, designing an effective network architecture is time-consuming, and highly relies on specialist’s experience and prior knowledge. To address this issue, we introduce a searchable segmentation network (denoted as Auto-DenseUNet) based on the neural architecture search (NAS) to search the optimal architecture automatically for the ABUS mass segmentation task. Concretely, a novel search space is designed based on a densely connected structure to enhance the gradient and information flows throughout the network. Then, to encourage multiscale information fusion, a set of searchable multiscale aggregation nodes between the down-sampling and up-sampling parts of the network are further designed. Thus, all the operators within the dense connection structure or between any two aggregation nodes can be searched to find the optimal structure. Finally, a novel decoupled search training strategy during architecture search is also introduced to alleviate the memory limitation caused by continuous relaxation in NAS. The proposed Auto-DeseUNet method has been evaluated on our ABUS dataset with 170 volumes (from 107 patients), including 120 training volumes and 50 testing volumes split at patient level. Experimental results on testing volumes show that our searched architecture performed better than several human-designed segmentation models on the 3D ABUS mass segmentation task, indicating the effectiveness of our proposed method.
1889.
Yuan, Jun; Liu, Mengchen; Tian, Fengyuan; Liu, Shixia
Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-09665,
title = {Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles},
author = {Jun Yuan and Mengchen Liu and Fengyuan Tian and Shixia Liu},
url = {https://doi.org/10.48550/arXiv.2208.09665},
doi = {10.48550/arXiv.2208.09665},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.09665},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1888.
Xu, Shuhan; Ren, Yudan; Tao, Zeyang; Song, Limei; He, Xiaowei
In: eNeuro, 2022.
@article{XuENEURO.0200-22.2022,
title = {Hierarchical Individual Naturalistic Functional Brain Networks with Group Consistency uncovered by a Two-Stage NAS-Volumetric Sparse DBN Framework},
author = {Shuhan Xu and Yudan Ren and Zeyang Tao and Limei Song and Xiaowei He},
url = {https://www.eneuro.org/content/early/2022/08/19/ENEURO.0200-22.2022},
doi = {10.1523/ENEURO.0200-22.2022},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {eNeuro},
publisher = {Society for Neuroscience},
abstract = {The functional magnetic resonance imaging under naturalistic paradigm (NfMRI) showed great advantages in identifying complex and interactive functional brain networks due to its dynamics and multimodal information. In recent years, various deep learning models, such as deep convolutional autoencoder (DCAE), deep belief network (DBN) and volumetric sparse deep belief network (vsDBN), can obtain hierarchical functional brain networks (FBN) and temporal features from fMRI data. Among them, the vsDBN model revealed a good capability in identifying hierarchical FBNs by modelling fMRI volume images. However, due to the high dimensionality of fMRI volumes and the diverse training parameters of deep learning methods, especially the network architecture that is the most critical parameter for uncovering the hierarchical organization of human brain function, researchers still face challenges in designing an appropriate deep learning framework with automatic network architecture optimization to model volumetric NfMRI. In addition, most of the existing deep learning models ignore the group-wise consistency and inter-subject variation properties embedded in NfMRI volumes. To solve these problems, we proposed a two-stage neural architecture search and vs DBN model (two-stage NAS-vsDBN model) to identify the hierarchical human brain spatio-temporal features possessing both group-consistency and individual-uniqueness under naturalistic condition. Moreover, our model defined reliable network structure for modelling volumetric NfMRI data via NAS framework, and the group-level and individual-level FBNs and associated temporal features exhibited great consistency. In general, our method well identified the hierarchical temporal and spatial features of the brain function and revealed the crucial properties of neural processes under natural viewing condition.Significance StatementIn this paper, we proposed and applied a novel analytical strategy – a two-stage NAS-vsDBN model to identify both group-level and individual-level spatio-temporal features at multi-scales from volumetric NfMRI data. The proposed PSO-based NAS framework can find optimal neural structure for both group-wise and individual-level vs-DBN models. Furthermore, with well-established correspondence between two stages of vsDBN models, our model can effectively detect group-level FBNs that reveal the consistency in neural processes across subjects and individual-level FBNs that maintain the subject specific variability, verifying the inherent property of brain function under naturalistic condition.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The functional magnetic resonance imaging under naturalistic paradigm (NfMRI) showed great advantages in identifying complex and interactive functional brain networks due to its dynamics and multimodal information. In recent years, various deep learning models, such as deep convolutional autoencoder (DCAE), deep belief network (DBN) and volumetric sparse deep belief network (vsDBN), can obtain hierarchical functional brain networks (FBN) and temporal features from fMRI data. Among them, the vsDBN model revealed a good capability in identifying hierarchical FBNs by modelling fMRI volume images. However, due to the high dimensionality of fMRI volumes and the diverse training parameters of deep learning methods, especially the network architecture that is the most critical parameter for uncovering the hierarchical organization of human brain function, researchers still face challenges in designing an appropriate deep learning framework with automatic network architecture optimization to model volumetric NfMRI. In addition, most of the existing deep learning models ignore the group-wise consistency and inter-subject variation properties embedded in NfMRI volumes. To solve these problems, we proposed a two-stage neural architecture search and vs DBN model (two-stage NAS-vsDBN model) to identify the hierarchical human brain spatio-temporal features possessing both group-consistency and individual-uniqueness under naturalistic condition. Moreover, our model defined reliable network structure for modelling volumetric NfMRI data via NAS framework, and the group-level and individual-level FBNs and associated temporal features exhibited great consistency. In general, our method well identified the hierarchical temporal and spatial features of the brain function and revealed the crucial properties of neural processes under natural viewing condition.Significance StatementIn this paper, we proposed and applied a novel analytical strategy – a two-stage NAS-vsDBN model to identify both group-level and individual-level spatio-temporal features at multi-scales from volumetric NfMRI data. The proposed PSO-based NAS framework can find optimal neural structure for both group-wise and individual-level vs-DBN models. Furthermore, with well-established correspondence between two stages of vsDBN models, our model can effectively detect group-level FBNs that reveal the consistency in neural processes across subjects and individual-level FBNs that maintain the subject specific variability, verifying the inherent property of brain function under naturalistic condition.
1887.
Li, Nan; Ma, Lianbo; Yu, Guo; Xue, Bing; Zhang, Mengjie; Jin, Yaochu
Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-10658,
title = {Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues},
author = {Nan Li and Lianbo Ma and Guo Yu and Bing Xue and Mengjie Zhang and Yaochu Jin},
url = {https://doi.org/10.48550/arXiv.2208.10658},
doi = {10.48550/arXiv.2208.10658},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.10658},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1886.
Ghosh, Arjun; Jana, Nanda Dulal; Mallik, Saurav; Zhao, Zhongming
Designing optimal convolutional neural network architecture using differential evolution algorithm Journal Article
In: Patterns, vol. 3, no. 9, pp. 100567, 2022, ISSN: 2666-3899.
@article{GHOSH2022100567,
title = {Designing optimal convolutional neural network architecture using differential evolution algorithm},
author = {Arjun Ghosh and Nanda Dulal Jana and Saurav Mallik and Zhongming Zhao},
url = {https://www.sciencedirect.com/science/article/pii/S2666389922001787},
doi = {https://doi.org/10.1016/j.patter.2022.100567},
issn = {2666-3899},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Patterns},
volume = {3},
number = {9},
pages = {100567},
abstract = {Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
1885.
Wang, Zihan; Wan, Chengcheng; Chen, Yuting; Lin, Ziyi; Jiang, He; Qiao, Lei
Hierarchical Memory-Constrained Operator Scheduling of Neural Architecture Search Networks Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 493–498, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530472,
title = {Hierarchical Memory-Constrained Operator Scheduling of Neural Architecture Search Networks},
author = {Zihan Wang and Chengcheng Wan and Yuting Chen and Ziyi Lin and He Jiang and Lei Qiao},
url = {https://doi.org/10.1145/3489517.3530472},
doi = {10.1145/3489517.3530472},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {493–498},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7~42.4% of peak memory footprints, and achieving 137--283x of speedups in scheduling.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7~42.4% of peak memory footprints, and achieving 137--283x of speedups in scheduling.
1884.
Negi, Shubham; Chakraborty, Indranil; Ankit, Aayush; Roy, Kaushik
NAX: Neural Architecture and Memristive Xbar Based Accelerator Co-Design Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 451–456, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530476,
title = {NAX: Neural Architecture and Memristive Xbar Based Accelerator Co-Design},
author = {Shubham Negi and Indranil Chakraborty and Aayush Ankit and Kaushik Roy},
url = {https://doi.org/10.1145/3489517.3530476},
doi = {10.1145/3489517.3530476},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {451–456},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Neural Architecture Search (NAS) has provided the ability to design efficient deep neural network (DNN) catered towards different hardwares like GPUs, CPUs etc. However, integrating NAS with Memristive Crossbar Array (MCA) based In-Memory Computing (IMC) accelerator remains an open problem. The hardware efficiency (energy, latency and area) as well as application accuracy (considering device and circuit non-idealities) of DNNs mapped to such hardware are co-dependent on network parameters such as kernel size, depth etc. and hardware architecture parameters such as crossbar size and the precision of analog-to-digital converters. Co-optimization of both network and hardware parameters presents a challenging search space comprising of different kernel sizes mapped to varying crossbar sizes. To that effect, we propose NAX - an efficient neural architecture search engine that co-designs neural network and IMC based hardware architecture. NAX explores the aforementioned search space to determine kernel and corresponding crossbar sizes for each DNN layer to achieve optimal tradeoffs between hardware efficiency and application accuracy. For CIFAR-10 and Tiny ImageNet, our models achieve 0.9% and 18.57% higher accuracy at 30% and -10.47% lower EDAP (energy-delay-area product), compared to baseline ResNet-20 and ResNet-18 models, respectively.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) has provided the ability to design efficient deep neural network (DNN) catered towards different hardwares like GPUs, CPUs etc. However, integrating NAS with Memristive Crossbar Array (MCA) based In-Memory Computing (IMC) accelerator remains an open problem. The hardware efficiency (energy, latency and area) as well as application accuracy (considering device and circuit non-idealities) of DNNs mapped to such hardware are co-dependent on network parameters such as kernel size, depth etc. and hardware architecture parameters such as crossbar size and the precision of analog-to-digital converters. Co-optimization of both network and hardware parameters presents a challenging search space comprising of different kernel sizes mapped to varying crossbar sizes. To that effect, we propose NAX - an efficient neural architecture search engine that co-designs neural network and IMC based hardware architecture. NAX explores the aforementioned search space to determine kernel and corresponding crossbar sizes for each DNN layer to achieve optimal tradeoffs between hardware efficiency and application accuracy. For CIFAR-10 and Tiny ImageNet, our models achieve 0.9% and 18.57% higher accuracy at 30% and -10.47% lower EDAP (energy-delay-area product), compared to baseline ResNet-20 and ResNet-18 models, respectively.
1883.
Luo, Xiangzhong; Liu, Di; Kong, Hao; Huai, Shuo; Chen, Hui; Liu, Weichen
You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 475–480, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530488,
title = {You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms},
author = {Xiangzhong Luo and Di Liu and Hao Kong and Shuo Huai and Hui Chen and Weichen Liu},
url = {https://doi.org/10.1145/3489517.3530488},
doi = {10.1145/3489517.3530488},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {475–480},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., you only search once). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods. Related codes will be released at https://github.com/stepbuystep/LightNAS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., you only search once). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods. Related codes will be released at https://github.com/stepbuystep/LightNAS.
1882.
Jääsaari, Elias; Ma, Michelle; Talwalkar, Ameet; Chen, Tianqi
SONAR: Joint Architecture and System Optimization Search Technical Report
2022.
@techreport{jaasaari2022sonar,
title = {SONAR: Joint Architecture and System Optimization Search},
author = {Elias Jääsaari and Michelle Ma and Ameet Talwalkar and Tianqi Chen},
url = {https://arxiv.org/abs/2208.12218},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {arXiv preprint arXiv:2208.12218},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1881.
Hong, Deokki; Choi, Kanghyun; Lee, Hye Yoon; Yu, Joonsang; Park, Noseong; Kim, Youngsok; Lee, Jinho
Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 589–594, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530507,
title = {Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration},
author = {Deokki Hong and Kanghyun Choi and Hye Yoon Lee and Joonsang Yu and Noseong Park and Youngsok Kim and Jinho Lee},
url = {https://doi.org/10.1145/3489517.3530507},
doi = {10.1145/3489517.3530507},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {589–594},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems. The large co-exploration space is often handled by adopting the idea of differentiable neural architecture search. However, despite the superior search efficiency of the differentiable co-exploration, it faces a critical challenge of not being able to systematically satisfy hard constraints such as frame rate. To handle the hard constraint problem of differentiable co-exploration, we propose HDX, which searches for hard-constrained solutions without compromising the global design objectives. By manipulating the gradients in the interest of the given hard constraint, high-quality solutions satisfying the constraint can be obtained.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems. The large co-exploration space is often handled by adopting the idea of differentiable neural architecture search. However, despite the superior search efficiency of the differentiable co-exploration, it faces a critical challenge of not being able to systematically satisfy hard constraints such as frame rate. To handle the hard constraint problem of differentiable co-exploration, we propose HDX, which searches for hard-constrained solutions without compromising the global design objectives. By manipulating the gradients in the interest of the given hard constraint, high-quality solutions satisfying the constraint can be obtained.
1880.
Ghosh, Arjun; Jana, Nanda Dulal; Mallik, Saurav; Zhao, Zhongming
Designing optimal convolutional neural network architecture using differential evolution algorithm Journal Article
In: Patterns, vol. 3, no. 9, pp. 100567, 2022, ISSN: 2666-3899.
@article{GHOSH2022100567b,
title = {Designing optimal convolutional neural network architecture using differential evolution algorithm},
author = {Arjun Ghosh and Nanda Dulal Jana and Saurav Mallik and Zhongming Zhao},
url = {https://www.sciencedirect.com/science/article/pii/S2666389922001787},
doi = {https://doi.org/10.1016/j.patter.2022.100567},
issn = {2666-3899},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Patterns},
volume = {3},
number = {9},
pages = {100567},
abstract = {Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
1879.
Chen, Yantong; Bian, Shichang; Liu, Yang; Zhang, Zhongling
A novel method based on neural architecture search for Diptera insect classification on embedded devices Journal Article
In: Ecological Informatics, vol. 71, pp. 101791, 2022, ISSN: 1574-9541.
@article{CHEN2022101791,
title = {A novel method based on neural architecture search for Diptera insect classification on embedded devices},
author = {Yantong Chen and Shichang Bian and Yang Liu and Zhongling Zhang},
url = {https://www.sciencedirect.com/science/article/pii/S1574954122002412},
doi = {https://doi.org/10.1016/j.ecoinf.2022.101791},
issn = {1574-9541},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Ecological Informatics},
volume = {71},
pages = {101791},
abstract = {Diptera insects have the characteristics of spreading diseases and destroying forests. There are similarities among different species, which makes it difficult to identify a Diptera insect. Most traditional convolutional neural networks have large parameters and high recognition latency. Therefore, they are not suitable for deploying models on embedded devices for classification and recognition. This paper proposes an improved neural architecture based on differentiable search method. First, we designed a network search cell by adding the feature output of the previous layer to each search cell. Second, we added the attention module to the search space to expand the searchable range. At the same time, we used methods such as model quantization and limiting the ReLU function to the ReLU6 function to reduce computer resource consumption. Finally, the network model was transplanted to the NVIDIA Jetson Xavier NX embedded development platform to verify the network performance so that the neural architecture search could be organically combined with the embedded development platform. The experimental results show that the designed neural architecture achieves 98.9% accuracy on the Diptera insect dataset with a latency of 8.4 ms. It has important practical significance for the recognition of Diptera insects in embedded devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Diptera insects have the characteristics of spreading diseases and destroying forests. There are similarities among different species, which makes it difficult to identify a Diptera insect. Most traditional convolutional neural networks have large parameters and high recognition latency. Therefore, they are not suitable for deploying models on embedded devices for classification and recognition. This paper proposes an improved neural architecture based on differentiable search method. First, we designed a network search cell by adding the feature output of the previous layer to each search cell. Second, we added the attention module to the search space to expand the searchable range. At the same time, we used methods such as model quantization and limiting the ReLU function to the ReLU6 function to reduce computer resource consumption. Finally, the network model was transplanted to the NVIDIA Jetson Xavier NX embedded development platform to verify the network performance so that the neural architecture search could be organically combined with the embedded development platform. The experimental results show that the designed neural architecture achieves 98.9% accuracy on the Diptera insect dataset with a latency of 8.4 ms. It has important practical significance for the recognition of Diptera insects in embedded devices.
1878.
Vikashini, Vetha; Salam, Hanan; Nasir, Jauwairia; Bruno, Barbara; Celiktutan, Oya
Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning Proceedings Article
In: 2022.
@inproceedings{Vikashini:296044,
title = {Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning},
author = {Vetha Vikashini and Hanan Salam and Jauwairia Nasir and Barbara Bruno and Oya Celiktutan},
url = {http://infoscience.epfl.ch/record/296044},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
abstract = {In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profiles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identified which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profiles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose: (1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profile are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners' profiles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant's behaviours in fixed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the recognition performance with respect to non-personalized models when training models for the gainer vs. non-gainer groups, (2) multitask NAS (implicit personalization) also outperforms non-personalized models, (3) the speech modality has high contribution towards prediction, and (4) non-incremental features outperform the incremental ones overall.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profiles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identified which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profiles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose: (1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profile are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners' profiles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant's behaviours in fixed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the recognition performance with respect to non-personalized models when training models for the gainer vs. non-gainer groups, (2) multitask NAS (implicit personalization) also outperforms non-personalized models, (3) the speech modality has high contribution towards prediction, and (4) non-incremental features outperform the incremental ones overall.
1877.
Shimizu, Shoma; Nishio, Takayuki; Saito, Shota; Hirose, Yoichi; Chen, Yen-Hsiu; Shirakawa, Shinichi
Neural Architecture Search for Improving Latency-Accuracy Trade-off in Split Computing Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-13968,
title = {Neural Architecture Search for Improving Latency-Accuracy Trade-off in Split Computing},
author = {Shoma Shimizu and Takayuki Nishio and Shota Saito and Yoichi Hirose and Yen-Hsiu Chen and Shinichi Shirakawa},
url = {https://doi.org/10.48550/arXiv.2208.13968},
doi = {10.48550/arXiv.2208.13968},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.13968},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1876.
Yakovlev, Konstantin D.; Grebenkova, Olga S.; Bakhteev, Oleg Y.; Strijov, Vadim V.
Neural Architecture Search with Structure Complexity Control Proceedings Article
In: Burnaev, Evgeny; Ignatov, Dmitry I.; Ivanov, Sergei; Khachay, Michael; Koltsova, Olessia; Kutuzov, Andrei; Kuznetsov, Sergei O.; Loukachevitch, Natalia; Napoli, Amedeo; Panchenko, Alexander; Pardalos, Panos M.; Saramäki, Jari; Savchenko, Andrey V.; Tsymbalov, Evgenii; Tutubalina, Elena (Ed.): Recent Trends in Analysis of Images, Social Networks and Texts, pp. 207–219, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-15168-2.
@inproceedings{10.1007/978-3-031-15168-2_17,
title = {Neural Architecture Search with Structure Complexity Control},
author = {Konstantin D. Yakovlev and Olga S. Grebenkova and Oleg Y. Bakhteev and Vadim V. Strijov},
editor = {Evgeny Burnaev and Dmitry I. Ignatov and Sergei Ivanov and Michael Khachay and Olessia Koltsova and Andrei Kutuzov and Sergei O. Kuznetsov and Natalia Loukachevitch and Amedeo Napoli and Alexander Panchenko and Panos M. Pardalos and Jari Saramäki and Andrey V. Savchenko and Evgenii Tsymbalov and Elena Tutubalina},
isbn = {978-3-031-15168-2},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Recent Trends in Analysis of Images, Social Networks and Texts},
pages = {207--219},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {The paper investigates the problem of deep learning model selection. We propose a method of a neural architecture search with respect to the desired model complexity called DARTS-CC. An amount of parameters in the model is considered as a model complexity. The proposed method is based on a differential architecture search algorithm (DARTS). Instead of optimizing structural parameters of the architecture, we consider them as a function depending on the complexity parameter. It enables us to obtain multiple architectures at one optimization procedure and select the architecture based on our computation budget. To evaluate the performance of the proposed algorithm, we conduct experiments on the Fashion-MNIST and CIFAR-10 datasets and compare the resulting architecture with architectures obtained by other neural architecture search methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The paper investigates the problem of deep learning model selection. We propose a method of a neural architecture search with respect to the desired model complexity called DARTS-CC. An amount of parameters in the model is considered as a model complexity. The proposed method is based on a differential architecture search algorithm (DARTS). Instead of optimizing structural parameters of the architecture, we consider them as a function depending on the complexity parameter. It enables us to obtain multiple architectures at one optimization procedure and select the architecture based on our computation budget. To evaluate the performance of the proposed algorithm, we conduct experiments on the Fashion-MNIST and CIFAR-10 datasets and compare the resulting architecture with architectures obtained by other neural architecture search methods.
1875.
Xue, Boyang; Hu, Shoukang; Xu, Junhao; Geng, Mengzhe; Liu, Xunying; Meng, Helen
Bayesian Neural Network Language Modeling for Speech Recognition Journal Article
In: IEEE ACM Trans. Audio Speech Lang. Process., vol. 30, pp. 2900–2917, 2022.
@article{DBLP:journals/taslp/XueHXGLM22,
title = {Bayesian Neural Network Language Modeling for Speech Recognition},
author = {Boyang Xue and Shoukang Hu and Junhao Xu and Mengzhe Geng and Xunying Liu and Helen Meng},
url = {https://doi.org/10.1109/TASLP.2022.3203891},
doi = {10.1109/TASLP.2022.3203891},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
volume = {30},
pages = {2900--2917},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1874.
Yin, Benshun; Chen, Zhiyong; Tao, Meixia
Dynamic Data Collection and Neural Architecture Search for Wireless Edge Intelligence Systems Journal Article
In: IEEE Transactions on Wireless Communications, pp. 1-1, 2022.
@article{9861242,
title = {Dynamic Data Collection and Neural Architecture Search for Wireless Edge Intelligence Systems},
author = {Benshun Yin and Zhiyong Chen and Meixia Tao},
url = {https://ieeexplore.ieee.org/abstract/document/9861242},
doi = {10.1109/TWC.2022.3197809},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Wireless Communications},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1873.
Han, Zhu; Hong, Danfeng; Gao, Lianru; Roy, Swalpa Kumar; Zhang, Bing; Chanussot, Jocelyn
Reinforcement Learning for Neural Architecture Search in Hyperspectral Unmixing Journal Article
In: IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022.
@article{9865216,
title = {Reinforcement Learning for Neural Architecture Search in Hyperspectral Unmixing},
author = {Zhu Han and Danfeng Hong and Lianru Gao and Swalpa Kumar Roy and Bing Zhang and Jocelyn Chanussot},
url = {https://ieeexplore.ieee.org/abstract/document/9865216},
doi = {10.1109/LGRS.2022.3199583},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Geoscience and Remote Sensing Letters},
volume = {19},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1872.
Xu, Peng; Wang, Ke; Hassan, Mohammad Mehedi; Chen, Chien-Ming; Lin, Weiguo; Hassan, Md. Rafiul; Fortino, Giancarlo
Adversarial Robustness in Graph-Based Neural Architecture Search for Edge AI Transportation Systems Journal Article
In: IEEE Transactions on Intelligent Transportation Systems, pp. 1-10, 2022.
@article{9868259,
title = {Adversarial Robustness in Graph-Based Neural Architecture Search for Edge AI Transportation Systems},
author = {Peng Xu and Ke Wang and Mohammad Mehedi Hassan and Chien-Ming Chen and Weiguo Lin and Md. Rafiul Hassan and Giancarlo Fortino},
url = {https://ieeexplore.ieee.org/abstract/document/9868259},
doi = {10.1109/TITS.2022.3197713},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Intelligent Transportation Systems},
pages = {1-10},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1871.
Shvetsov, Egor; Osin, Dmitry; Zaytsev, Alexey; Koryakovskiy, Ivan; Buchnev, Valentin; Trofimov, Ilya; Burnaev, Evgeny
QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-14839,
title = {QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise},
author = {Egor Shvetsov and Dmitry Osin and Alexey Zaytsev and Ivan Koryakovskiy and Valentin Buchnev and Ilya Trofimov and Evgeny Burnaev},
url = {https://doi.org/10.48550/arXiv.2208.14839},
doi = {10.48550/arXiv.2208.14839},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.14839},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1870.
Cotrim, Lucas P.; Barreira, Rodrigo A.; Santos, Ismael H. F.; Gomi, Edson S.; Costa, Anna Helena Reali; Tannuri, Eduardo Aoun
Neural Network Meta-Models for FPSO Motion Prediction From Environmental Data With Different Platform Loads Journal Article
In: IEEE Access, vol. 10, pp. 86558–86577, 2022.
@article{DBLP:journals/access/CotrimBSGCT22,
title = {Neural Network Meta-Models for FPSO Motion Prediction From Environmental Data With Different Platform Loads},
author = {Lucas P. Cotrim and Rodrigo A. Barreira and Ismael H. F. Santos and Edson S. Gomi and Anna Helena Reali Costa and Eduardo Aoun Tannuri},
url = {https://doi.org/10.1109/ACCESS.2022.3199009},
doi = {10.1109/ACCESS.2022.3199009},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Access},
volume = {10},
pages = {86558--86577},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1869.
Zhang, Li Lyna; Homma, Youkow; Wang, Yujing; Wu, Min; Yang, Mao; Zhang, Ruofei; Cao, Ting; Shen, Wei
SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-00625,
title = {SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance},
author = {Li Lyna Zhang and Youkow Homma and Yujing Wang and Min Wu and Mao Yang and Ruofei Zhang and Ting Cao and Wei Shen},
url = {https://doi.org/10.48550/arXiv.2209.00625},
doi = {10.48550/arXiv.2209.00625},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.00625},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1868.
Liang, Jingkang; Liao, Yixiao; Chen, Zhuyun; Lin, Huibin; Jin, Gang; Gryllias, Konstantinos; Li, Weihua
Intelligent fault diagnosis of rotating machinery using lightweight network with modified tree-structured parzen estimators Journal Article
In: IET Collaborative Intelligent Manufacturing, vol. 4, no. 3, pp. 194-207, 2022.
@article{https://doi.org/10.1049/cim2.12055,
title = {Intelligent fault diagnosis of rotating machinery using lightweight network with modified tree-structured parzen estimators},
author = {Jingkang Liang and Yixiao Liao and Zhuyun Chen and Huibin Lin and Gang Jin and Konstantinos Gryllias and Weihua Li},
url = {https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/cim2.12055},
doi = {https://doi.org/10.1049/cim2.12055},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IET Collaborative Intelligent Manufacturing},
volume = {4},
number = {3},
pages = {194-207},
abstract = {Abstract Deep learning-based methods have been widely used in the field of rotating machinery fault diagnosis. It is of practical significance to improve the calculation speed of the model on the premise of ensuring accuracy, so as to realise real-time fault diagnosis. However, designing an efficient and lightweight fault diagnosis network requires expert knowledge to determine the network structure and adjust the hyperparameters of the network, which is time-consuming and laborious. In order to design fault diagnosis networks considering both time and accuracy effortlessly, a novel lightweight network with modified tree-structured parzen estimators (LN-MT) is proposed for intelligent fault diagnosis of rotating machinery. Firstly, a lightweight framework based on global average pooling and group convolution is proposed, and a hyperparameter optimisation (HPO) method based on Bayesian optimisation called tree-structured parzen estimator is utilised to automatically search the optimal hyperparameters for the fault diagnosis task. The objective of the HPO algorithm is the weighting of accuracy and calculating time, so as to find models that balance both time and accuracy. The results of comparison experiments indicate that LN-MT can achieve superior fault diagnosis accuracies with few trainable parameters and less calculating time.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Deep learning-based methods have been widely used in the field of rotating machinery fault diagnosis. It is of practical significance to improve the calculation speed of the model on the premise of ensuring accuracy, so as to realise real-time fault diagnosis. However, designing an efficient and lightweight fault diagnosis network requires expert knowledge to determine the network structure and adjust the hyperparameters of the network, which is time-consuming and laborious. In order to design fault diagnosis networks considering both time and accuracy effortlessly, a novel lightweight network with modified tree-structured parzen estimators (LN-MT) is proposed for intelligent fault diagnosis of rotating machinery. Firstly, a lightweight framework based on global average pooling and group convolution is proposed, and a hyperparameter optimisation (HPO) method based on Bayesian optimisation called tree-structured parzen estimator is utilised to automatically search the optimal hyperparameters for the fault diagnosis task. The objective of the HPO algorithm is the weighting of accuracy and calculating time, so as to find models that balance both time and accuracy. The results of comparison experiments indicate that LN-MT can achieve superior fault diagnosis accuracies with few trainable parameters and less calculating time.
1867.
Mandal, Murari; Meedimale, Yashwanth Reddy; Reddy, M. Satish Kumar; Vipparthi, Santosh Kumar
Neural Architecture Search for Image Dehazing Journal Article
In: IEEE Transactions on Artificial Intelligence, pp. 1-11, 2022.
@article{9878218,
title = {Neural Architecture Search for Image Dehazing},
author = {Murari Mandal and Yashwanth Reddy Meedimale and M. Satish Kumar Reddy and Santosh Kumar Vipparthi},
url = {https://ieeexplore.ieee.org/abstract/document/9878218},
doi = {10.1109/TAI.2022.3204732},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Artificial Intelligence},
pages = {1-11},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1866.
Zoljodi, Ali; Loni, Mohammad; Abadijou, Sadegh; Alibeigi, Mina; Daneshtalab, Masoud
3DLaneNAS: Neural Architecture Search for Accurate and Light-Weight 3D Lane Detection Proceedings Article
In: Pimenidis, Elias; Angelov, Plamen; Jayne, Chrisina; Papaleonidas, Antonios; Aydin, Mehmet (Ed.): Ärtificial Neural Networks and Machine Learning -- ICANN 2022", pp. 404–415, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-15919-0.
@inproceedings{10.1007/978-3-031-15919-0_34,
title = {3DLaneNAS: Neural Architecture Search for Accurate and Light-Weight 3D Lane Detection},
author = {Ali Zoljodi and Mohammad Loni and Sadegh Abadijou and Mina Alibeigi and Masoud Daneshtalab},
editor = {Elias Pimenidis and Plamen Angelov and Chrisina Jayne and Antonios Papaleonidas and Mehmet Aydin},
isbn = {978-3-031-15919-0},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Ärtificial Neural Networks and Machine Learning -- ICANN 2022"},
pages = {404--415},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Lane detection is one of the most fundamental tasks for autonomous driving. It plays a crucial role in the lateral control and the precise localization of autonomous vehicles. Monocular 3D lane detection methods provide state-of-the-art results for estimating the position of lanes in 3D world coordinates using only the information obtained from the front-view camera. Recent advances in Neural Architecture Search (NAS) facilitate automated optimization of various computer vision tasks. NAS can automatically optimize monocular 3D lane detection methods to enhance the extraction and combination of visual features, consequently reducing computation loads and increasing accuracy. This paper proposes 3DLaneNAS, a multi-objective method that enhances the accuracy of monocular 3D lane detection for both short- and long-distance scenarios while at the same time providing a fair amount of hardware acceleration. 3DLaneNAS utilizes a new multi-objective energy function to optimize the architecture of feature extraction and feature fusion modules simultaneously. Moreover, a transfer learning mechanism is used to improve the convergence of the search process. Experimental results reveal that 3DLaneNAS yields a minimum of 5.2% higher accuracy and $$backslashapprox $$≈1.33$$backslashtimes $$texttimeslower latency over competing methods on the synthetic-3D-lanes dataset. Code is at https://github.com/alizoljodi/3DLaneNAS},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Lane detection is one of the most fundamental tasks for autonomous driving. It plays a crucial role in the lateral control and the precise localization of autonomous vehicles. Monocular 3D lane detection methods provide state-of-the-art results for estimating the position of lanes in 3D world coordinates using only the information obtained from the front-view camera. Recent advances in Neural Architecture Search (NAS) facilitate automated optimization of various computer vision tasks. NAS can automatically optimize monocular 3D lane detection methods to enhance the extraction and combination of visual features, consequently reducing computation loads and increasing accuracy. This paper proposes 3DLaneNAS, a multi-objective method that enhances the accuracy of monocular 3D lane detection for both short- and long-distance scenarios while at the same time providing a fair amount of hardware acceleration. 3DLaneNAS utilizes a new multi-objective energy function to optimize the architecture of feature extraction and feature fusion modules simultaneously. Moreover, a transfer learning mechanism is used to improve the convergence of the search process. Experimental results reveal that 3DLaneNAS yields a minimum of 5.2% higher accuracy and $$backslashapprox $$≈1.33$$backslashtimes $$texttimeslower latency over competing methods on the synthetic-3D-lanes dataset. Code is at https://github.com/alizoljodi/3DLaneNAS