Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
5555
2208.
Chen, J.; Gao, J.; Lyu, T.; Oloulade, B.; Hu, X.
AutoMSR: Auto Molecular Structure Representation Learning for Multi-label Metabolic Pathway Prediction Journal Article
In: IEEE/ACM Transactions on Computational Biology and Bioinformatics, no. 01, pp. 1-11, 5555, ISSN: 1557-9964.
@article{9864145,
title = {AutoMSR: Auto Molecular Structure Representation Learning for Multi-label Metabolic Pathway Prediction},
author = {J. Chen and J. Gao and T. Lyu and B. Oloulade and X. Hu},
doi = {10.1109/TCBB.2022.3198119},
issn = {1557-9964},
year = {5555},
date = {5555-08-01},
urldate = {5555-08-01},
journal = {IEEE/ACM Transactions on Computational Biology and Bioinformatics},
number = {01},
pages = {1-11},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {It is significant to comprehend the relationship between metabolic pathway and molecular pathway for synthesizing new molecules, for instance optimizing drug metabolization. In bioinformatics fields, multi-label prediction of metabolic pathways is a typical manner to understand this relationship. Graph neural networks (GNNs) have become an effective method to extract molecular structure's features for multi-label prediction of metabolic pathways. Though GNNs can effectively capture structural features from molecular structure graphs, building a well-performed GNN model for a given molecular structure data set requires the manual design of the GNN architecture and fine-tuning of the hyperparameters, which are time-consuming and rely on expert experience. To address the above challenge, we design an end-to-end automatic molecular structure representation learning framework named AutoMSR that can design the optimal GNN model based on a given molecular structure data set without manual intervention. We propose a multi-seed age evolution (MSAE) search algorithm to identify the optimal GNN architecture from the GNN architecture subspace. For a given molecular structure data set, AutoMSR first uses MSAE to search the GNN architecture, and then it adopts a tree-structured parzen estimator to obtain the best hyperparameters in the hyperparameters subspace. Finally, AutoMSR automatically constructs the optimal GNN model based on the best GNN architecture and hyperparameters to extract the molecular structure features for multi-label metabolic pathway prediction. We test the performance of AutoMSR on the real data set KEGG. The experiment results show that AutoMSR outperforms baseline methods on different multi-label classification evaluation metrics. keywords = computer architecture;task analysis;feature extraction;graph neural networks;prediction algorithms;drugs;architecture},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
It is significant to comprehend the relationship between metabolic pathway and molecular pathway for synthesizing new molecules, for instance optimizing drug metabolization. In bioinformatics fields, multi-label prediction of metabolic pathways is a typical manner to understand this relationship. Graph neural networks (GNNs) have become an effective method to extract molecular structure's features for multi-label prediction of metabolic pathways. Though GNNs can effectively capture structural features from molecular structure graphs, building a well-performed GNN model for a given molecular structure data set requires the manual design of the GNN architecture and fine-tuning of the hyperparameters, which are time-consuming and rely on expert experience. To address the above challenge, we design an end-to-end automatic molecular structure representation learning framework named AutoMSR that can design the optimal GNN model based on a given molecular structure data set without manual intervention. We propose a multi-seed age evolution (MSAE) search algorithm to identify the optimal GNN architecture from the GNN architecture subspace. For a given molecular structure data set, AutoMSR first uses MSAE to search the GNN architecture, and then it adopts a tree-structured parzen estimator to obtain the best hyperparameters in the hyperparameters subspace. Finally, AutoMSR automatically constructs the optimal GNN model based on the best GNN architecture and hyperparameters to extract the molecular structure features for multi-label metabolic pathway prediction. We test the performance of AutoMSR on the real data set KEGG. The experiment results show that AutoMSR outperforms baseline methods on different multi-label classification evaluation metrics. keywords = computer architecture;task analysis;feature extraction;graph neural networks;prediction algorithms;drugs;architecture
2023
2207.
Kolganov, Pavel A.; Tiumentsev, Yury V.
Än Attempt to Formalize the Formulation of the Network Architecture Search Problem for Convolutional Neural Networks Inproceedings
In: Kryzhanovsky, Boris; Dunin-Barkowski, Witali; Redko, Vladimir; Tiumentsev, Yury (Ed.): Ädvances in Neural Computation, Machine Learning, and Cognitive Research VI", pp. 550–556, Springer International Publishing, Cham, 2023, ISBN: 978-3-031-19032-2.
@inproceedings{10.1007/978-3-031-19032-2_55,
title = {Än Attempt to Formalize the Formulation of the Network Architecture Search Problem for Convolutional Neural Networks},
author = {Pavel A. Kolganov and Yury V. Tiumentsev},
editor = {Boris Kryzhanovsky and Witali Dunin-Barkowski and Vladimir Redko and Yury Tiumentsev},
isbn = {978-3-031-19032-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ädvances in Neural Computation, Machine Learning, and Cognitive Research VI"},
pages = {550--556},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {The paper deals with the problem of searching for a neural network architecture. The paper presents a mathematical formulation of the problem of searching a neural network model, optimal from the point of view of a predefined criterion. The analysis of components of this problem is given. Some difficulties encountered by researchers when solving the NAS problem are described. A computational experiment is conducted, which consists in the search of a neural network architecture on the MNIST dataset.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The paper deals with the problem of searching for a neural network architecture. The paper presents a mathematical formulation of the problem of searching a neural network model, optimal from the point of view of a predefined criterion. The analysis of components of this problem is given. Some difficulties encountered by researchers when solving the NAS problem are described. A computational experiment is conducted, which consists in the search of a neural network architecture on the MNIST dataset.
2206.
Zharikov, Ilia; Krivorotov, Ivan; Maximov, Egor; Korviakov, Vladimir; Letunovskiy, Alexey
Ä Review of One-Shot Neural Architecture Search Methods Inproceedings
In: Kryzhanovsky, Boris; Dunin-Barkowski, Witali; Redko, Vladimir; Tiumentsev, Yury (Ed.): Ädvances in Neural Computation, Machine Learning, and Cognitive Research VI", pp. 130–147, Springer International Publishing, Cham, 2023, ISBN: 978-3-031-19032-2.
@inproceedings{10.1007/978-3-031-19032-2_14,
title = {Ä Review of One-Shot Neural Architecture Search Methods},
author = {Ilia Zharikov and Ivan Krivorotov and Egor Maximov and Vladimir Korviakov and Alexey Letunovskiy},
editor = {Boris Kryzhanovsky and Witali Dunin-Barkowski and Vladimir Redko and Yury Tiumentsev},
isbn = {978-3-031-19032-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ädvances in Neural Computation, Machine Learning, and Cognitive Research VI"},
pages = {130--147},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Neural network architecture design is a challenging and computational expensive problem. For this reason training a one-shot model becomes very popular way to obtain several architectures or find the best according to different requirements without retraining. In this paper we summarize the existing one-shot NAS methods, highlight base concepts and compare considered methods in terms of accuracy, number of needed for training GPU hours and ranking quality.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural network architecture design is a challenging and computational expensive problem. For this reason training a one-shot model becomes very popular way to obtain several architectures or find the best according to different requirements without retraining. In this paper we summarize the existing one-shot NAS methods, highlight base concepts and compare considered methods in terms of accuracy, number of needed for training GPU hours and ranking quality.
2205.
Li, Chuanyou; Zhang, Kun; Li, Yifan; Shang, Jiangwei; Zhang, Xinyue; Qian, Lei
ANNA: Accelerating Neural Network Accelerator through software-hardware co-design for vertical applications in edge systems Journal Article
In: Future Generation Computer Systems, vol. 140, pp. 91-103, 2023, ISSN: 0167-739X.
@article{LI202391,
title = {ANNA: Accelerating Neural Network Accelerator through software-hardware co-design for vertical applications in edge systems},
author = {Chuanyou Li and Kun Zhang and Yifan Li and Jiangwei Shang and Xinyue Zhang and Lei Qian},
url = {https://www.sciencedirect.com/science/article/pii/S0167739X22003168},
doi = {https://doi.org/10.1016/j.future.2022.10.001},
issn = {0167-739X},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Future Generation Computer Systems},
volume = {140},
pages = {91-103},
abstract = {In promising edge systems, AI algorithms and their hardware implementations are often joint optimized as integrated solutions to solve end-to-end design problems. Joint optimization depends on a delicate co-design of software and hardware. According to our knowledge, current co-design methodologies are still coarse-grained. In this paper, we proposed ANNA: Accelerating Neural Network Accelerator through a novel software-hardware co-design methodology. ANNA is a framework composed of three components: ANNA-NAS (Neural Architecture Search), ANNA-ARCH (hardware ARCHitecture) and ANNA-PERF (PERFormance optimizer & evaluator). ANNA-NAS adopts a cell-wise structure and is designed to be hardware aware. It aims at generating a neural network having high inference accuracy and low inference latency. To avoid tremendous time costs, ANNA-NAS synthetically uses differentiable architecture search and early stopping techniques. ANNA-ARCH starts to be designed as long as the architecture search space is defined. Based on the cell-wise structure, ANNA-ARCH specifies its main body which includes Convolution units, Activation Router and Buffer Pool. To well support different neural networks that could be generated by ANNA-NAS, the detailed part of ANNA-ARCH is configurable. ANNA-PERF harmonizes the co-design of ANNA-NAS and ANNA-ARCH. It takes a neural network and a hardware architecture as inputs. After optimizing the mapping strategy between the neural network and hardware accelerator, it feeds back a cycle-accurate latency to ANNA-NAS. Aiming at image classification, we carried out the experiments on ImageNet. Experimental results demonstrate that without loss of much inference accuracy, ANNA wins a significant low inference latency through a harmonious software and hardware co-design.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In promising edge systems, AI algorithms and their hardware implementations are often joint optimized as integrated solutions to solve end-to-end design problems. Joint optimization depends on a delicate co-design of software and hardware. According to our knowledge, current co-design methodologies are still coarse-grained. In this paper, we proposed ANNA: Accelerating Neural Network Accelerator through a novel software-hardware co-design methodology. ANNA is a framework composed of three components: ANNA-NAS (Neural Architecture Search), ANNA-ARCH (hardware ARCHitecture) and ANNA-PERF (PERFormance optimizer & evaluator). ANNA-NAS adopts a cell-wise structure and is designed to be hardware aware. It aims at generating a neural network having high inference accuracy and low inference latency. To avoid tremendous time costs, ANNA-NAS synthetically uses differentiable architecture search and early stopping techniques. ANNA-ARCH starts to be designed as long as the architecture search space is defined. Based on the cell-wise structure, ANNA-ARCH specifies its main body which includes Convolution units, Activation Router and Buffer Pool. To well support different neural networks that could be generated by ANNA-NAS, the detailed part of ANNA-ARCH is configurable. ANNA-PERF harmonizes the co-design of ANNA-NAS and ANNA-ARCH. It takes a neural network and a hardware architecture as inputs. After optimizing the mapping strategy between the neural network and hardware accelerator, it feeds back a cycle-accurate latency to ANNA-NAS. Aiming at image classification, we carried out the experiments on ImageNet. Experimental results demonstrate that without loss of much inference accuracy, ANNA wins a significant low inference latency through a harmonious software and hardware co-design.
2204.
Wen, Yingpeng; Yu, Weijiang; Li, Dongsheng; Du, Jiangsu; Huang, Dan; Xiao, Nong
CosNAS: Enhancing estimation on cosmological parameters via neural architecture search Journal Article
In: New Astronomy, vol. 99, pp. 101955, 2023, ISSN: 1384-1076.
@article{WEN2023101955,
title = {CosNAS: Enhancing estimation on cosmological parameters via neural architecture search},
author = {Yingpeng Wen and Weijiang Yu and Dongsheng Li and Jiangsu Du and Dan Huang and Nong Xiao},
url = {https://www.sciencedirect.com/science/article/pii/S1384107622001373},
doi = {https://doi.org/10.1016/j.newast.2022.101955},
issn = {1384-1076},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {New Astronomy},
volume = {99},
pages = {101955},
abstract = {A great challenge of cosmology is estimating the cosmological parameters of the universe. With the development of deep learning, scientists adopt 3D deep neural networks to estimate cosmological parameters from the large-scale dark matter distribution of the universe, but these methods are time-consuming to design and train neural networks. While neural architecture search is an emerging approach to estimate cosmological parameters with its capability of automatically designing neural networks, the 3D operations on a 3D dataset prohibit the usage of traditional neural architecture search methods, due to its overwhelming time and memory consumption. To tackle these issues, we propose an efficient method, CosNAS, that can automatically design neural networks with 2D operations to estimate the cosmological parameters. In addition, processing 3D data with 2D operations will inevitably cause the loss of spatial information, thus we propose an efficient SABlock to retain more 3D spatial information. We also customize a space-focused search space to focus on important information in the dark matter distribution. The experimental results indicate that our estimation of the cosmological parameters Ω, σ and n, can be applied to large-scale 3D dark matter distribution and speedup the network search by 800x. The average relative errors of cosmological parameter estimations are (0.00163, 0.00065, 0.00080), significantly decreasing the average error of estimation by 85.5% compared to previous work.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
A great challenge of cosmology is estimating the cosmological parameters of the universe. With the development of deep learning, scientists adopt 3D deep neural networks to estimate cosmological parameters from the large-scale dark matter distribution of the universe, but these methods are time-consuming to design and train neural networks. While neural architecture search is an emerging approach to estimate cosmological parameters with its capability of automatically designing neural networks, the 3D operations on a 3D dataset prohibit the usage of traditional neural architecture search methods, due to its overwhelming time and memory consumption. To tackle these issues, we propose an efficient method, CosNAS, that can automatically design neural networks with 2D operations to estimate the cosmological parameters. In addition, processing 3D data with 2D operations will inevitably cause the loss of spatial information, thus we propose an efficient SABlock to retain more 3D spatial information. We also customize a space-focused search space to focus on important information in the dark matter distribution. The experimental results indicate that our estimation of the cosmological parameters Ω, σ and n, can be applied to large-scale 3D dark matter distribution and speedup the network search by 800x. The average relative errors of cosmological parameter estimations are (0.00163, 0.00065, 0.00080), significantly decreasing the average error of estimation by 85.5% compared to previous work.
2203.
Souquet, Léo; Shvai, Nadiya; Llanza, Arcadi; Nakib, Amir
Convolutional neural network architecture search based on fractal decomposition optimization algorithm Journal Article
In: Expert Systems with Applications, vol. 213, pp. 118947, 2023, ISSN: 0957-4174.
@article{SOUQUET2023118947,
title = {Convolutional neural network architecture search based on fractal decomposition optimization algorithm},
author = {Léo Souquet and Nadiya Shvai and Arcadi Llanza and Amir Nakib},
url = {https://www.sciencedirect.com/science/article/pii/S0957417422019650},
doi = {https://doi.org/10.1016/j.eswa.2022.118947},
issn = {0957-4174},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Expert Systems with Applications},
volume = {213},
pages = {118947},
abstract = {This paper presents a new approach to design the architecture and optimize the hyperparameters of a deep convolutional neural network (CNN) via of the Fractal Decomposition Algorithm (FDA). This optimization algorithm was recently proposed to solve continuous optimization problems. It is based on a geometric fractal decomposition that divides the search space while searching for the best solution possible. As FDA is effective in single-objective optimization, in this work we aim to prove that it can also be successfully applied to fine-tuning deep neural network architectures. Moreover, a new formulation based on bi-level optimization is proposed to separate the architecture search composed of discrete parameters from hyperparameters’ optimization. This is motivated by the fact that automating the construction of deep neural architecture has been an important focus over recent years as manual construction is considerably time-consuming, error-prone, and requires in-depth knowledge. To solve the bi-level problem thus formulated, a random search is performed aiming to create a set of candidate architectures. Then, the best ones are finetuned using FDA. CIFAR-10 and CIFAR-100 benchmarks were used to evaluate the performance of the proposed approach. The results obtained are among the state of the art in the corresponding class of networks (low number of parameters and chained-structured CNN architectures). The results are emphasized by the fact that the whole process was performed using low computing power with only 3 NVIDIA V100 GPUs. The source code is available at https://github.com/alc1218/Convolutional-Neural-Network-Architecture-Search-Based-on-Fractal-Decomposition-Optimization.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper presents a new approach to design the architecture and optimize the hyperparameters of a deep convolutional neural network (CNN) via of the Fractal Decomposition Algorithm (FDA). This optimization algorithm was recently proposed to solve continuous optimization problems. It is based on a geometric fractal decomposition that divides the search space while searching for the best solution possible. As FDA is effective in single-objective optimization, in this work we aim to prove that it can also be successfully applied to fine-tuning deep neural network architectures. Moreover, a new formulation based on bi-level optimization is proposed to separate the architecture search composed of discrete parameters from hyperparameters’ optimization. This is motivated by the fact that automating the construction of deep neural architecture has been an important focus over recent years as manual construction is considerably time-consuming, error-prone, and requires in-depth knowledge. To solve the bi-level problem thus formulated, a random search is performed aiming to create a set of candidate architectures. Then, the best ones are finetuned using FDA. CIFAR-10 and CIFAR-100 benchmarks were used to evaluate the performance of the proposed approach. The results obtained are among the state of the art in the corresponding class of networks (low number of parameters and chained-structured CNN architectures). The results are emphasized by the fact that the whole process was performed using low computing power with only 3 NVIDIA V100 GPUs. The source code is available at https://github.com/alc1218/Convolutional-Neural-Network-Architecture-Search-Based-on-Fractal-Decomposition-Optimization.
2202.
Deng, TianJin; Wu, Jia
Efficient graph neural architecture search using Monte Carlo Tree search and prediction network Journal Article
In: Expert Systems with Applications, vol. 213, pp. 118916, 2023, ISSN: 0957-4174.
@article{DENG2023118916,
title = {Efficient graph neural architecture search using Monte Carlo Tree search and prediction network},
author = {TianJin Deng and Jia Wu},
url = {https://www.sciencedirect.com/science/article/pii/S0957417422019340},
doi = {https://doi.org/10.1016/j.eswa.2022.118916},
issn = {0957-4174},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Expert Systems with Applications},
volume = {213},
pages = {118916},
abstract = {Graph Neural Networks (GNNs) have emerged recently as a powerful way of dealing with non-Euclidean data on graphs, such as social networks and citation networks. Despite their success, obtaining optimal graph neural networks requires immense manual work and domain knowledge. Inspired by the strong searching capability of neural architecture search in CNN, a few attempts automatically search optimal GNNs that rival the best human-invented architectures. However, existing Graph Neural Architecture Search (GNAS) approaches face two challenges: (1) Sampling GNNs across the entire search space results in low search efficiency, particularly in large search spaces. (2) It is pretty costly to evaluate GNNs by training architectures from scratch. To overcome these challenges, this paper proposes an Efficient Graph Neural Architecture Search (EGNAS) method based on Monte Carlo Tree Search (MCTS) and a prediction network. Specifically, EGNAS first uses MCTS to recursively partition the entire search space into good or bad search regions. Then, the reinforcement learning-based search strategy (also called the agent) is applied to sample GNNs in those good search regions, which prevents overly exploring complex architectures and bad-performance regions, thus improving sampling efficiency. To reduce the evaluation cost, we use a prediction network to estimate the performance of GNNs. We alternately use ground-truth accuracy (by training GNNs from scratch) and prediction accuracy (by the prediction network) to update the search strategy to avoid inaccuracies caused by long-term use of the prediction network. Furthermore, to improve the training efficiency and stability, the agent is trained by a variant of Proximal Policy Optimization. Experiments show that EGNAS can search for better GNNs in the promising search region in a shorter search time, with an accuracy of 83.5%, 73.3%, 79.6%, and 94.5% on Cora, Citeseer, Pubmed, and Photo datasets, respectively In particular, compared to the most popular GNAS algorithm, our EGNAS-NP without using the prediction network achieves an accuracy of 83.6% on Cora, 73.5% on Citeseer, 79.9% on Pubmed, and 94.6% on Photo, with a relative improvement of 0.6%, 0.2%, 0.7%, and 0.6%.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Graph Neural Networks (GNNs) have emerged recently as a powerful way of dealing with non-Euclidean data on graphs, such as social networks and citation networks. Despite their success, obtaining optimal graph neural networks requires immense manual work and domain knowledge. Inspired by the strong searching capability of neural architecture search in CNN, a few attempts automatically search optimal GNNs that rival the best human-invented architectures. However, existing Graph Neural Architecture Search (GNAS) approaches face two challenges: (1) Sampling GNNs across the entire search space results in low search efficiency, particularly in large search spaces. (2) It is pretty costly to evaluate GNNs by training architectures from scratch. To overcome these challenges, this paper proposes an Efficient Graph Neural Architecture Search (EGNAS) method based on Monte Carlo Tree Search (MCTS) and a prediction network. Specifically, EGNAS first uses MCTS to recursively partition the entire search space into good or bad search regions. Then, the reinforcement learning-based search strategy (also called the agent) is applied to sample GNNs in those good search regions, which prevents overly exploring complex architectures and bad-performance regions, thus improving sampling efficiency. To reduce the evaluation cost, we use a prediction network to estimate the performance of GNNs. We alternately use ground-truth accuracy (by training GNNs from scratch) and prediction accuracy (by the prediction network) to update the search strategy to avoid inaccuracies caused by long-term use of the prediction network. Furthermore, to improve the training efficiency and stability, the agent is trained by a variant of Proximal Policy Optimization. Experiments show that EGNAS can search for better GNNs in the promising search region in a shorter search time, with an accuracy of 83.5%, 73.3%, 79.6%, and 94.5% on Cora, Citeseer, Pubmed, and Photo datasets, respectively In particular, compared to the most popular GNAS algorithm, our EGNAS-NP without using the prediction network achieves an accuracy of 83.6% on Cora, 73.5% on Citeseer, 79.9% on Pubmed, and 94.6% on Photo, with a relative improvement of 0.6%, 0.2%, 0.7%, and 0.6%.
2201.
Liu, Yang; Liang, Xinle; Luo, Jiahuan; He, Yuanqin; Chen, Tianjian; Yao, Quanming; Yang, Qiang
Cross-Silo Federated Neural Architecture Search for Heterogeneous and Cooperative Systems Book Chapter
In: Razavi-Far, Roozbeh; Wang, Boyu; Taylor, Matthew E.; Yang, Qiang (Ed.): Federated and Transfer Learning, pp. 57–86, Springer International Publishing, Cham, 2023, ISBN: 978-3-031-11748-0.
@inbook{Liu2023,
title = {Cross-Silo Federated Neural Architecture Search for Heterogeneous and Cooperative Systems},
author = {Yang Liu and Xinle Liang and Jiahuan Luo and Yuanqin He and Tianjian Chen and Quanming Yao and Qiang Yang},
editor = {Roozbeh Razavi-Far and Boyu Wang and Matthew E. Taylor and Qiang Yang},
url = {https://doi.org/10.1007/978-3-031-11748-0_4},
doi = {10.1007/978-3-031-11748-0_4},
isbn = {978-3-031-11748-0},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Federated and Transfer Learning},
pages = {57--86},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {In many cooperative systems (i.e. autonomous vehicles, robotics, hospital networks), data are privately and heterogeneously distributed among devices with various computational constraints, and no party has a global view of data or device distribution. Federated Neural Architecture Search (FedNAS) was previously proposed to adapt Neural Architecture Search (NAS) into Federated Learning (FL) to provide both privacy and model performance to such uninspectable and heterogeneous systems. However, these approaches mostly apply to scenarios where parties share the same data attributes and comparable computation resources. In this chapter, we present Self-supervised Vertical Federated Neural Architecture Search (SS-VFNAS) for automating FL where participants have heterogeneous data and resource constraints, a common cross-silo scenario. SS-VFNAS not only simultaneously optimizes all parties' model architecture and parameters for the best global performance under a vertical FL (VFL) framework using only a small set of aligned and labeled data, but also preserves each party's local optimal model architecture under a self-supervised NAS framework. We demonstrate that SS-VFNAS is a promising framework of superior performance, communication efficiency and privacy, and is capable of generating high-performance and highly-transferable heterogeneous architectures with only limited overlapping samples, providing practical solutions for designing collaborative systems with both limited data and resource constraints.},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
In many cooperative systems (i.e. autonomous vehicles, robotics, hospital networks), data are privately and heterogeneously distributed among devices with various computational constraints, and no party has a global view of data or device distribution. Federated Neural Architecture Search (FedNAS) was previously proposed to adapt Neural Architecture Search (NAS) into Federated Learning (FL) to provide both privacy and model performance to such uninspectable and heterogeneous systems. However, these approaches mostly apply to scenarios where parties share the same data attributes and comparable computation resources. In this chapter, we present Self-supervised Vertical Federated Neural Architecture Search (SS-VFNAS) for automating FL where participants have heterogeneous data and resource constraints, a common cross-silo scenario. SS-VFNAS not only simultaneously optimizes all parties' model architecture and parameters for the best global performance under a vertical FL (VFL) framework using only a small set of aligned and labeled data, but also preserves each party's local optimal model architecture under a self-supervised NAS framework. We demonstrate that SS-VFNAS is a promising framework of superior performance, communication efficiency and privacy, and is capable of generating high-performance and highly-transferable heterogeneous architectures with only limited overlapping samples, providing practical solutions for designing collaborative systems with both limited data and resource constraints.
2200.
Liang, Jingkang; Liao, Yixiao; Li, Weihua
Differentiable Architecture Searched Network with Tree-Structured Parzen Estimators for Rotating Machinery Fault Diagnosis Inproceedings
In: Zhang, Hao; Feng, Guojin; Wang, Hongjun; Gu, Fengshou; Sinha, Jyoti K. (Ed.): Proceedings of IncoME-VI and TEPEN 2021, pp. 959–970, Springer International Publishing, Cham, 2023, ISBN: 978-3-030-99075-6.
@inproceedings{10.1007/978-3-030-99075-6_77,
title = {Differentiable Architecture Searched Network with Tree-Structured Parzen Estimators for Rotating Machinery Fault Diagnosis},
author = {Jingkang Liang and Yixiao Liao and Weihua Li},
editor = {Hao Zhang and Guojin Feng and Hongjun Wang and Fengshou Gu and Jyoti K. Sinha},
isbn = {978-3-030-99075-6},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of IncoME-VI and TEPEN 2021},
pages = {959--970},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Deep learning is widely used in the field of rotating machinery fault diagnosis. However, manually designing the neural network structure and adjusting the hyperparameters for specific fault diagnosis task are complex and requires a lot of expert knowledge. Aiming at these problems, Differentiable Architecture Searched Network with Tree-Structured Parzen Estimators (DASNT) is proposed for fault diagnosis. Differentiable Architecture Search (DARTS) is utilized to automatically search network structure for specific fault diagnosis task. Tree-Structured Parzen Estimators (TPE) is utilized to optimize the hyperparameters of the network searched by DARTS, which can further improve the fault diagnosis accuracy. The results of comparison experiments indicate that the network architecture and hyperparameters optimized by DASNT can achieve superior fault diagnosis performance.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Deep learning is widely used in the field of rotating machinery fault diagnosis. However, manually designing the neural network structure and adjusting the hyperparameters for specific fault diagnosis task are complex and requires a lot of expert knowledge. Aiming at these problems, Differentiable Architecture Searched Network with Tree-Structured Parzen Estimators (DASNT) is proposed for fault diagnosis. Differentiable Architecture Search (DARTS) is utilized to automatically search network structure for specific fault diagnosis task. Tree-Structured Parzen Estimators (TPE) is utilized to optimize the hyperparameters of the network searched by DARTS, which can further improve the fault diagnosis accuracy. The results of comparison experiments indicate that the network architecture and hyperparameters optimized by DASNT can achieve superior fault diagnosis performance.
2199.
Rajarajeswari, S.; Patil, Annapurna P.; Madhyastha, Aditya; Jaitly, Akshat; Jha, Himangshu Shekhar; Bhave, Sahil Rajesh; Das, Mayukh; Pradeep, N. S.
Design and Develop Hardware Aware DNN for Faster Inference Inproceedings
In: Ärai, Kohei" (Ed.): Intelligent Systems and Applications, pp. 309–318, Springer International Publishing, Cham, 2023, ISBN: 978-3-031-16075-2.
@inproceedings{10.1007/978-3-031-16075-2_21,
title = {Design and Develop Hardware Aware DNN for Faster Inference},
author = {S. Rajarajeswari and Annapurna P. Patil and Aditya Madhyastha and Akshat Jaitly and Himangshu Shekhar Jha and Sahil Rajesh Bhave and Mayukh Das and N. S. Pradeep},
editor = {Kohei" Ärai},
isbn = {978-3-031-16075-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Intelligent Systems and Applications},
pages = {309--318},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Ön many small-scale devices, advanced learning models have become standard. The necessity of the hour is to reduce the amount of time required for inference. This study describes a pipeline for automating Deep Neural Network customization and reducing neural network inference time. This paper presents a hardware-aware methodology in the form of a sequential pipeline for shrinking the size of deep neural networks. MorphNet is used at the pipeline's core to iteratively decrease and enlarge a network. Upon the activation of layers, a resource-weighted sparsifying regularizer is used to identify and prune inefficient neurons, and all layers are then expanded using a uniform multiplicative factor. This is followed by fusion, a technique for combining the frozen batch normalization layer with the preceding convolution layer. Finally, the DNN is retrained after customization using a Knowledge Distillation approach to maintain model accuracy performance. The approach shows promising initial results on MobileNetv1 and ResNet50 architectures."},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Ön many small-scale devices, advanced learning models have become standard. The necessity of the hour is to reduce the amount of time required for inference. This study describes a pipeline for automating Deep Neural Network customization and reducing neural network inference time. This paper presents a hardware-aware methodology in the form of a sequential pipeline for shrinking the size of deep neural networks. MorphNet is used at the pipeline's core to iteratively decrease and enlarge a network. Upon the activation of layers, a resource-weighted sparsifying regularizer is used to identify and prune inefficient neurons, and all layers are then expanded using a uniform multiplicative factor. This is followed by fusion, a technique for combining the frozen batch normalization layer with the preceding convolution layer. Finally, the DNN is retrained after customization using a Knowledge Distillation approach to maintain model accuracy performance. The approach shows promising initial results on MobileNetv1 and ResNet50 architectures."
2198.
Wang, Ziyan; Qi, Feng; Zou, Liming
Continuous Evolution for Efficient Neural Architecture Search Based on Improved NSGA-III Algorithm Inproceedings
In: Sun, Jiande; Wang, Yue; Huo, Mengyao; Xu, Lexi (Ed.): Signal and Information Processing, Networking and Computers, pp. 979–986, Springer Nature Singapore, Singapore, 2023, ISBN: 978-981-19-3387-5.
@inproceedings{10.1007/978-981-19-3387-5_117,
title = {Continuous Evolution for Efficient Neural Architecture Search Based on Improved NSGA-III Algorithm},
author = {Ziyan Wang and Feng Qi and Liming Zou},
editor = {Jiande Sun and Yue Wang and Mengyao Huo and Lexi Xu},
isbn = {978-981-19-3387-5},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Signal and Information Processing, Networking and Computers},
pages = {979--986},
publisher = {Springer Nature Singapore},
address = {Singapore},
abstract = {Improved Continuous Evolution for Efficient Neural Architecture Search method (I-CARS) is proposed to solve multi-objective optimization problems (MOPs). Seeking to improve the convergence and search accuracy, two modifications of the non-dominated sorting genetic algorithm based on reference-point strategy (NSGA-III) were made to replace pNSGA-III in CARS, including the penalty-based boundary intersection distance (PBI distance) and the selection-and-elimination operators. First, the perpendicular distance from solutions to reference lines was replaced by the PBI distance in the offspring selection stage phase, which can add the convergence information. Second, individuals in the population were selected or eliminated by niche count and PBI distance. Better performing individuals were selected to become the next generation, while poorly performing individuals were eliminated. The convergence and diversity of the population can be balanced by adding the selection-and-elimination operator. Experiments were conducted on Vega pipeline, and I-CARS achieves 3.41% test error on CIFAR10, the results indicated that the accuracy and convergence of I-CARS are enhanced compared to CARS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Improved Continuous Evolution for Efficient Neural Architecture Search method (I-CARS) is proposed to solve multi-objective optimization problems (MOPs). Seeking to improve the convergence and search accuracy, two modifications of the non-dominated sorting genetic algorithm based on reference-point strategy (NSGA-III) were made to replace pNSGA-III in CARS, including the penalty-based boundary intersection distance (PBI distance) and the selection-and-elimination operators. First, the perpendicular distance from solutions to reference lines was replaced by the PBI distance in the offspring selection stage phase, which can add the convergence information. Second, individuals in the population were selected or eliminated by niche count and PBI distance. Better performing individuals were selected to become the next generation, while poorly performing individuals were eliminated. The convergence and diversity of the population can be balanced by adding the selection-and-elimination operator. Experiments were conducted on Vega pipeline, and I-CARS achieves 3.41% test error on CIFAR10, the results indicated that the accuracy and convergence of I-CARS are enhanced compared to CARS.
2197.
Franchini, Giorgia; Ruggiero, Valeria; Porta, Federica; Zanni, Luca
Neural architecture search via standard machine learning methodologies Journal Article
In: Mathematics in Engineering, 2023.
@article{Franchini2023,
title = {Neural architecture search via standard machine learning methodologies},
author = {Giorgia Franchini and Valeria Ruggiero and Federica Porta and Luca Zanni},
url = {https://www.aimspress.com/article/doi/10.3934/mine.2023012?viewType=HTML},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Mathematics in Engineering},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2022
2196.
Lu, Bingqian
Towards Ultra-Efficient Machine Learning for Edge Inference PhD Thesis
2022.
@phdthesis{LuPhD2022,
title = {Towards Ultra-Efficient Machine Learning for Edge Inference},
author = {Bingqian Lu},
url = {https://escholarship.org/uc/item/0wg9692z#main},
year = {2022},
date = {2022-11-01},
urldate = {2022-11-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2195.
Park, Jae Bok; Lee, Kyung Hee; Kwak, Ji Young; Cho, Chang Sik
Deployment Framework Design Techniques for Optimized Neural Network Applicati Technical Manual
2022.
@manual{ParkICC22,
title = {Deployment Framework Design Techniques for Optimized Neural Network Applicati},
author = {Jae Bok Park and Kyung Hee Lee and Ji Young Kwak and Chang Sik Cho},
url = {https://journal-home.s3.ap-northeast-2.amazonaws.com/site/ictc2022/abs/P7-4.pdf},
year = {2022},
date = {2022-11-01},
urldate = {2022-11-01},
keywords = {},
pubstate = {published},
tppubtype = {manual}
}
2194.
Nowak, Aleksandra I.; Janik, Romuald A.
Discovering wiring patterns influencing neural network performance Inproceedings
In: ECML PKDD 22, 2022.
@inproceedings{NowakECML22,
title = {Discovering wiring patterns influencing neural network performance},
author = {Aleksandra I. Nowak and Romuald A. Janik},
url = {https://2022.ecmlpkdd.org/wp-content/uploads/2022/09/sub_1358.pdf},
year = {2022},
date = {2022-09-19},
urldate = {2022-09-19},
booktitle = {ECML PKDD 22},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2193.
Stolle, Kurt; Vogel, Sebastian; van der Sommen, Fons; Sanberg, Willem
Block-Level Surrogate Models for Inference Time Estimation in Hardware-Aware Neural Architecture Search Inproceedings
In: ECML PKDD 22, 2022.
@inproceedings{StolleECML22,
title = {Block-Level Surrogate Models for Inference Time Estimation in Hardware-Aware Neural Architecture Search},
author = {Kurt Stolle and Sebastian Vogel and Fons van der Sommen and Willem Sanberg},
url = {https://2022.ecmlpkdd.org/wp-content/uploads/2022/09/sub_737.pdf},
year = {2022},
date = {2022-09-19},
urldate = {2022-09-19},
booktitle = {ECML PKDD 22},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2192.
Bhowmik, Pritom
Machine Learning in Production: From Experimented ML Model to System Journal Article
In: Journal of Robotics and Automation Research, 2022.
@article{Bhowmik22,
title = {Machine Learning in Production: From Experimented ML Model to System},
author = {Pritom Bhowmik},
url = {https://opastpublishers.com/open-access/machine-learning-in-production-from-experimented-ml-model-to-system.pdf},
year = {2022},
date = {2022-09-10},
urldate = {2022-09-10},
journal = {Journal of Robotics and Automation Research},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2191.
Yuan, Wanqi; Fu, Chenping; Liu, Risheng; Fan, Xin
SSoB: searching a scene-oriented architecture for underwater object detection Journal Article
In: The Visual Computer, 2022.
@article{YuanTVC22,
title = {SSoB: searching a scene-oriented architecture for underwater object detection},
author = {
Wanqi Yuan and Chenping Fu and Risheng Liu and Xin Fan
},
url = {https://link.springer.com/article/10.1007/s00371-022-02654-4},
year = {2022},
date = {2022-09-10},
urldate = {2022-09-10},
journal = {The Visual Computer},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2190.
Li, Yilan; Lu, Yantao; Cui, Helei; Velipasalar, Senem
Improving robustness and efficiency of edge computing models Journal Article
In: Wireless Networks , 2022.
@article{LiWirelessNetworks22,
title = {Improving robustness and efficiency of edge computing models},
author = {
Yilan Li and Yantao Lu and Helei Cui and Senem Velipasalar
},
url = {https://link.springer.com/article/10.1007/s11276-022-03115-5},
year = {2022},
date = {2022-09-05},
urldate = {2022-09-05},
journal = {Wireless Networks },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2189.
Arhore, Edore G.; Yasaee, Mehdi; Dayyani, Iman
Optimisation of convolutional neural network architecture using genetic algorithm for the prediction of adhesively bonded joint strength Journal Article
In: Structural and Multidisciplinary Optimization, 2022.
@article{ArhoreSMO22,
title = {Optimisation of convolutional neural network architecture using genetic algorithm for the prediction of adhesively bonded joint strength},
author = {
Edore G. Arhore and Mehdi Yasaee and Iman Dayyani
},
url = {https://link.springer.com/article/10.1007/s00158-022-03359-x},
year = {2022},
date = {2022-09-02},
urldate = {2022-09-02},
journal = {Structural and Multidisciplinary Optimization},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2188.
Liu, Yang; Liu, Jing; Li, Yifan
Automatic search of architecture and hyperparameters of graph convolutional networks for node classification Journal Article
In: Applied Intelligence , 2022.
@article{LiuAppliedIntelligence2022,
title = {Automatic search of architecture and hyperparameters of graph convolutional networks for node classification},
author = {
Yang Liu and Jing Liu and Yifan Li
},
url = {https://link.springer.com/article/10.1007/s10489-022-04096-w},
year = {2022},
date = {2022-08-31},
urldate = {2022-08-31},
booktitle = {Applied Intelligence },
journal = {Applied Intelligence },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2187.
Phan, Quan Minh; Luong, Ngoc Hoang
Enhancing multi-objective evolutionary neural architecture search with training-free Pareto local search Journal Article
In: Applied Intelligence, 2022.
@article{PhanApppliedInteligence22,
title = {Enhancing multi-objective evolutionary neural architecture search with training-free Pareto local search},
author = {
Quan Minh Phan and Ngoc Hoang Luong
},
url = {https://link.springer.com/article/10.1007/s10489-022-04032-y},
year = {2022},
date = {2022-08-24},
journal = {Applied Intelligence},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2186.
Fadzail, N F; Zali, S Mat; Mid1, E C; Jailani, R
Application of Automated Machine Learning (AutoML) Method in Wind Turbine Fault Detection Journal Article
In: Journal of Physics: Conference Series, 2022.
@article{FadzailJPCS2022,
title = {Application of Automated Machine Learning (AutoML) Method in Wind Turbine Fault Detection},
author = {N F Fadzail and S Mat Zali and E C Mid1 and R Jailani},
url = {https://iopscience.iop.org/article/10.1088/1742-6596/2312/1/012074/pdf},
year = {2022},
date = {2022-08-16},
urldate = {2022-08-16},
journal = {Journal of Physics: Conference Series},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2185.
ke, Songyu; Song, Li; Bao, Kainan; Pan, Zheyi; Zhang, Junbo; Zheng, Yu
EAST: An Enhanced Automated Machine Learning Library for Spatio-Temporal Forecasting Inproceedings
In: DeepSpatial ’22, Washington, DC, USA, 2022.
@inproceedings{KeEAST2022,
title = {EAST: An Enhanced Automated Machine Learning Library for Spatio-Temporal Forecasting},
author = {Songyu ke and Li Song and Kainan Bao and Zheyi Pan and Junbo Zhang and Yu Zheng},
year = {2022},
date = {2022-08-15},
urldate = {2022-08-15},
booktitle = {DeepSpatial ’22},
address = {Washington, DC, USA},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2184.
Bin, Wang; Hui, Ning
Notebook for PAN at CLEF 2022:Profiling Irony and Stereotype Spreaders on Twitter Inproceedings
In: CLEF 2022: Conference and Labs of the Evaluation Forum, 2022.
@inproceedings{BinPAN2022,
title = {Notebook for PAN at CLEF 2022:Profiling Irony and Stereotype Spreaders on Twitter},
author = {Wang Bin and Ning Hui},
url = {http://ceur-ws.org/Vol-3180/paper-225.pdf},
year = {2022},
date = {2022-08-12},
urldate = {2022-08-12},
booktitle = {CLEF 2022: Conference and Labs of the Evaluation Forum},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2183.
Arasakumaran, Umamageswari; Johnson, Shiny Duela; Sara, Dioline; Kothandaraman, Raja
An Enhanced Identification and Classification Algorithm for Plant Leaf Diseases Based on Deep Learning. Journal Article
In: Traitement du Signal, vol. 39, iss. 3, 2022.
@article{Umamageswari22,
title = { An Enhanced Identification and Classification Algorithm for Plant Leaf Diseases Based on Deep Learning. },
author = {Arasakumaran, Umamageswari and Johnson, Shiny Duela and Sara, Dioline and Kothandaraman, Raja },
url = {https://web.s.ebscohost.com/ehost/detail/detail?vid=0&sid=9bd58607-e82f-47ed-b179-299ab049b566%40redis&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#db=bsh&AN=158329091},
year = {2022},
date = {2022-08-10},
urldate = {2022-08-10},
journal = {Traitement du Signal},
volume = {39},
issue = {3},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2182.
Bechtel, M.; Weng, Q.; Yun, H.
DeepPicarMicro: Applying TinyML to Autonomous Cyber Physical Systems Inproceedings
In: 2022 IEEE 28th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), pp. 120-127, IEEE Computer Society, Los Alamitos, CA, USA, 2022.
@inproceedings{9904754,
title = {DeepPicarMicro: Applying TinyML to Autonomous Cyber Physical Systems},
author = {M. Bechtel and Q. Weng and H. Yun},
url = {https://doi.ieeecomputersociety.org/10.1109/RTCSA55878.2022.00019},
doi = {10.1109/RTCSA55878.2022.00019},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
booktitle = {2022 IEEE 28th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)},
pages = {120-127},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Running deep neural networks (DNNs) on tiny Micro-controller Units (MCUs) is challenging due to their limitations in computing, memory, and storage capacity. Fortunately, recent advances in both MCU hardware and machine learning software frameworks make it possible to run fairly complex neural networks on modern MCUs, resulting in a new field of study widely known as TinyML. However, there have been few studies to show the potential for TinyML applications in cyber physical systems (CPS).In this paper, we present DeepPicarMicro, a small self-driving RC car testbed, which runs a convolutional neural network (CNN) on a Raspberry Pi Pico MCU. We apply a state-of-the-art DNN optimization to successfully fit the well-known PilotNet CNN architecture, which was used to drive NVIDIA’s real self-driving car, on the MCU. We apply a state-of-art network architecture search (NAS) approach to find further optimized networks that can effectively control the car in real-time in an end-to-end manner. From an extensive systematic experimental evaluation study, we observe an interesting relationship between the accuracy, latency, and control performance of a system. From this, we propose a joint optimization strategy that takes both accuracy and latency of a model in the network architecture search process for AI enabled CPS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Running deep neural networks (DNNs) on tiny Micro-controller Units (MCUs) is challenging due to their limitations in computing, memory, and storage capacity. Fortunately, recent advances in both MCU hardware and machine learning software frameworks make it possible to run fairly complex neural networks on modern MCUs, resulting in a new field of study widely known as TinyML. However, there have been few studies to show the potential for TinyML applications in cyber physical systems (CPS).In this paper, we present DeepPicarMicro, a small self-driving RC car testbed, which runs a convolutional neural network (CNN) on a Raspberry Pi Pico MCU. We apply a state-of-the-art DNN optimization to successfully fit the well-known PilotNet CNN architecture, which was used to drive NVIDIA’s real self-driving car, on the MCU. We apply a state-of-art network architecture search (NAS) approach to find further optimized networks that can effectively control the car in real-time in an end-to-end manner. From an extensive systematic experimental evaluation study, we observe an interesting relationship between the accuracy, latency, and control performance of a system. From this, we propose a joint optimization strategy that takes both accuracy and latency of a model in the network architecture search process for AI enabled CPS.
2181.
Fakieh, Bahjat; Ragab, Mahmoud
Automated COVID-19 Classification Using Heap-Based Optimization with the Deep Transfer Learning Model Journal Article Forthcoming
In: Computational Intelligence and its Applications in Biomedical Engineering, vol. 2022, Forthcoming.
@article{FakiehCIN2022,
title = {Automated COVID-19 Classification Using Heap-Based Optimization with the Deep Transfer Learning Model},
author = {Bahjat Fakieh and Mahmoud Ragab},
url = {https://www.hindawi.com/journals/cin/2022/7508836/},
doi = { https://doi.org/10.1155/2022/7508836},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
journal = {Computational Intelligence and its Applications in Biomedical Engineering},
volume = {2022},
keywords = {},
pubstate = {forthcoming},
tppubtype = {article}
}
2180.
Turner, Jack
Efficient Neural Networks PhD Thesis
2022.
@phdthesis{TurnerPhD2022,
title = {Efficient Neural Networks},
author = {Jack Turner },
url = {https://era.ed.ac.uk/bitstream/handle/1842/39326/TurnerJ_2022.pdf?sequence=1&isAllowed=y},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2179.
Zheng, Kai; Zhao, Haochen; Zhao, Qichang; Wang, Bin; Gao, Xin; Wang, Jianxin
NASMDR: a framework for miRNA-drug resistance prediction using efficient neural architecture search and graph isomorphism networks Journal Article
In: Brief Bioinform., 2022.
@article{ZhengBriefBIio2022,
title = {NASMDR: a framework for miRNA-drug resistance prediction using efficient neural architecture search and graph isomorphism networks },
author = {Kai Zheng and Haochen Zhao and Qichang Zhao and Bin Wang and Xin Gao and Jianxin Wang},
url = {https://pubmed.ncbi.nlm.nih.gov/35998922/},
doi = {10.1093/bib/bbac338},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
journal = {Brief Bioinform.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2178.
Chen, Shengbo; Jiang, Kai; Liu, Xianrui; Yang, Kangkang; Lei, Zhou
TGAS-ReID: Efficient architecture search for person re-identification via greedy decisions with topological order Journal Article
In: Applied Intelligence , 2022.
@article{ChenAI2022,
title = {TGAS-ReID: Efficient architecture search for person re-identification via greedy decisions with topological order},
author = {Shengbo Chen and Kai Jiang and Xianrui Liu and Kangkang Yang and Zhou Lei
},
url = {https://link.springer.com/article/10.1007/s10489-021-03097-5},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
journal = {Applied Intelligence },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2177.
Li, Jolen; Galazis, Christopher; Popov, Illarion; Ovchinnikov, Lev; Vesnin, Sergey; Losev, Alexander; Goryanin, Igor
Dynamic Weight Agnostic Neural Networks and Medical Microwave Radiometry (MWR) for Breast Cancer Diagnostics Technical Report
2022.
@techreport{LiPrePrint2022,
title = {Dynamic Weight Agnostic Neural Networks and Medical Microwave Radiometry (MWR) for Breast Cancer Diagnostics},
author = { Jolen Li and Christopher Galazis and Illarion Popov and Lev Ovchinnikov and Sergey Vesnin and Alexander Losev and Igor Goryanin},
url = {https://www.preprints.org/manuscript/202207.0370/v1},
doi = {10.20944/preprints202207.0370.v1},
year = {2022},
date = {2022-07-29},
urldate = {2022-07-29},
howpublished = {Preprints 2022},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2176.
Peng, Cheng; Li, Yangyang; Shang, Ronghua; Jiao, Licheng
ReCNAS: Resource-Constrained Neural Architecture Search Based on Differentiable Annealing and Dynamic Pruning Journal Article
In: IEEE Trans Neural Netw Learn Syst , 2022.
@article{PengTNLS2022,
title = { ReCNAS: Resource-Constrained Neural Architecture Search Based on Differentiable Annealing and Dynamic Pruning },
author = {Cheng Peng and Yangyang Li and Ronghua Shang and Licheng Jiao
},
url = {https://pubmed.ncbi.nlm.nih.gov/35862327/},
doi = {10.1109/TNNLS.2022.3192169},
year = {2022},
date = {2022-07-26},
urldate = {2022-07-26},
journal = { IEEE Trans Neural Netw Learn Syst },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2175.
Jeong, Joonhyun; Yu, Joonsang; Han, Dongyoon; Yoo, YoungJoon
Neural Architecture Search with Loss Flatness-aware Measure Inproceedings
In: DyNN workshop at the 39th International Conference on Machine Learning, 2022.
@inproceedings{JeongDyNNICML2022,
title = {Neural Architecture Search with Loss Flatness-aware Measure},
author = {Joonhyun Jeong and Joonsang Yu and Dongyoon Han and YoungJoon Yoo},
url = {https://dynn-icml2022.github.io/papers/paper_11.pdf},
year = {2022},
date = {2022-07-26},
urldate = {2022-07-26},
booktitle = {DyNN workshop at the 39th International Conference on Machine Learning},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2174.
Chen, Panyue; Wang, Rui; Zhao, Ping; Liu, Guanming; Wei, Zhihua
Searching Efficient Dynamic Graph CNN for Point Cloud Processing Inproceedings
In: 1st International Conference on Automated Machine Learning, Late-Breaking Workshop, 2022.
@inproceedings{nokey,
title = {Searching Efficient Dynamic Graph CNN for Point Cloud Processing},
author = {Panyue Chen and Rui Wang and Ping Zhao and Guanming Liu and Zhihua Wei
},
url = {https://automl.cc/wp-content/uploads/2022/07/searching_efficient_dynamic_gr.pdf},
year = {2022},
date = {2022-07-21},
urldate = {2022-07-21},
booktitle = {1st International Conference on Automated Machine Learning, Late-Breaking Workshop},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2173.
Chatzimichailidis, Avraam; Zela, Arber; Keuper, Janis; Yang, Yang
GSparsity: Unifying Network Pruning and Neural Architecture Search by Group Sparsity Inproceedings
In: 1st International Conference on, Automated Machine Learning, Late-Breaking Workshop, 2022.
@inproceedings{Chatzimichailidisaml22,
title = {GSparsity: Unifying Network Pruning and Neural Architecture Search by Group Sparsity},
author = {Avraam Chatzimichailidis and Arber Zela and Janis Keuper and Yang Yang
},
url = {https://automl.cc/wp-content/uploads/2022/07/gsparsity_unifying_network_pru.pdf},
year = {2022},
date = {2022-07-21},
urldate = {2022-07-21},
booktitle = {1st International Conference on, Automated Machine Learning, Late-Breaking Workshop},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2172.
van Gastel, Rob; Vanschoren, Joaquin
Regularized Meta-Learning for Neural Architecture Search Inproceedings
In: 1st International Conference on, Automated Machine Learning, Late-Breaking Workshop, 2022.
@inproceedings{GastelAML22,
title = {Regularized Meta-Learning for Neural Architecture Search},
author = {Rob van Gastel and Joaquin Vanschoren
},
url = {https://automl.cc/wp-content/uploads/2022/07/regularized_meta_learning_for_.pdf},
year = {2022},
date = {2022-07-21},
urldate = {2022-07-21},
booktitle = {1st International Conference on, Automated Machine Learning, Late-Breaking Workshop},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2171.
Suchopárová, Gabriela; Neruda, Roman
Graph Embedding for Neural Architecture Search with Input-Output Information Inproceedings
In: 1st International Conference on, Automated Machine Learning, Late-Breaking Workshop, 2022.
@inproceedings{SuchopárováAML22,
title = {Graph Embedding for Neural Architecture Search with Input-Output Information},
author = {Gabriela Suchopárová and Roman Neruda},
url = {https://automl.cc/wp-content/uploads/2022/07/graph_embedding_for_neural_arc.pdf},
year = {2022},
date = {2022-07-21},
urldate = {2022-07-21},
booktitle = {1st International Conference on, Automated Machine Learning, Late-Breaking Workshop},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2170.
Zhang, Tianning; Kee, Chun Yun; Ang, Yee Sin; LI, Erping; Ang, Lay Kee
Symmetry Enhanced Network Architecture Search for Complex Metasurface Design Journal Article
In: IEEE Access, 2022.
@article{ZhangIEEEACCESS2022,
title = {Symmetry Enhanced Network Architecture Search for Complex Metasurface Design},
author = {Tianning Zhang and Chun Yun Kee and Yee Sin Ang and Erping LI and Lay Kee Ang},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9826756},
year = {2022},
date = {2022-07-18},
urldate = {2022-07-18},
journal = {IEEE Access},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2169.
Gupta, Abhibha; Sheth, Parth; Xie, Pengtao
Neural architecture search for pneumonia diagnosis from chest X-rays Journal Article
In: Scientific Reports, vol. 12, 2022.
@article{Gupta2022,
title = {Neural architecture search for pneumonia diagnosis from chest X-rays},
author = {Abhibha Gupta and Parth Sheth and Pengtao Xie
},
url = {https://www.nature.com/articles/s41598-022-15341-0},
year = {2022},
date = {2022-07-04},
urldate = {2022-07-04},
journal = {Scientific Reports},
volume = {12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2168.
Liao, W.; Zhang, Z.; Wang, X.; Yin, T.; Liu, H.; Ren, Z.; Yin, J.; Feng, S.
Distilldarts: Network Distillation for Smoothing Gradient Distributions in Differentiable Architecture Search Inproceedings
In: 2022 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, IEEE Computer Society, Los Alamitos, CA, USA, 2022.
@inproceedings{9859708,
title = {Distilldarts: Network Distillation for Smoothing Gradient Distributions in Differentiable Architecture Search},
author = {W. Liao and Z. Zhang and X. Wang and T. Yin and H. Liu and Z. Ren and J. Yin and S. Feng},
url = {https://doi.ieeecomputersociety.org/10.1109/ICME52920.2022.9859708},
doi = {10.1109/ICME52920.2022.9859708},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
booktitle = {2022 IEEE International Conference on Multimedia and Expo (ICME)},
pages = {1-6},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Recent studies show that differentiable architecture search (DARTS) suffers notable instability and collapse issue: skip-connect may gradually dominate the cell, leading to deteri-orating architectures. We conjecture that the domination of skip-connect is due to its superiority in gradient compen-sate. On this foundation, we propose a novel and stable method, called DistillDARTS, to stabilize DARTS by knowl-edge distillation and self-distillation scheme. Specifically, the distillation is able to serve as a substitute for skip-connect and smooth the back-propagated gradient distributions among layers of DARTS. By compensating gradients in shallow lay-ers, our method can relieve the dependence of gradient on skip-connect and hence mitigates the collapse issue. Exten-sive experiments on a range of benchmarks demonstrate that DistillDARTS can obtain sturdy architectures with few skip-connects without additional manual interventions, thus suc-cessfully improving the robustness of DARTS. Due to the im-proved stability, our proposed approach achieves the accuracy of 97.57% on CIFAR-10 and 75.8% on ImageNet.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Recent studies show that differentiable architecture search (DARTS) suffers notable instability and collapse issue: skip-connect may gradually dominate the cell, leading to deteri-orating architectures. We conjecture that the domination of skip-connect is due to its superiority in gradient compen-sate. On this foundation, we propose a novel and stable method, called DistillDARTS, to stabilize DARTS by knowl-edge distillation and self-distillation scheme. Specifically, the distillation is able to serve as a substitute for skip-connect and smooth the back-propagated gradient distributions among layers of DARTS. By compensating gradients in shallow lay-ers, our method can relieve the dependence of gradient on skip-connect and hence mitigates the collapse issue. Exten-sive experiments on a range of benchmarks demonstrate that DistillDARTS can obtain sturdy architectures with few skip-connects without additional manual interventions, thus suc-cessfully improving the robustness of DARTS. Due to the im-proved stability, our proposed approach achieves the accuracy of 97.57% on CIFAR-10 and 75.8% on ImageNet.
2167.
Poliakov, E.; Luu, V.; Tran, V.; Huang, C.
Model Compression via Structural Pruning and Feature Distillation for Accurate Multi-Spectral Object Detection on Edge-Devices Inproceedings
In: 2022 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, IEEE Computer Society, Los Alamitos, CA, USA, 2022.
@inproceedings{9859994,
title = {Model Compression via Structural Pruning and Feature Distillation for Accurate Multi-Spectral Object Detection on Edge-Devices},
author = {E. Poliakov and V. Luu and V. Tran and C. Huang},
url = {https://doi.ieeecomputersociety.org/10.1109/ICME52920.2022.9859994},
doi = {10.1109/ICME52920.2022.9859994},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
booktitle = {2022 IEEE International Conference on Multimedia and Expo (ICME)},
pages = {1-6},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Multi-spectral infrared object detection across different infrared wavelengths is a challenging task. Although some full-sized object detection models, such as YOLOv4 and ScaledYOLO, may achieve good infrared object detection, they are resource-demanding and unsuitable for real-time detection on edge devices. Tiny versions for object detection are proposed to meet the practical requirement, but they usually sacrifice model accuracy and generalization for efficiency. We propose an accurate and efficient object detector capable of performing real-time inference under the hardware constraints of an edge device by leveraging structural pruning, feature distillation, and neural architecture search (NAS). The experiments on FLIR and multi-spectral object detection datasets show that our model achieves comparable mAP to full-sized models while having 14x times fewer parameters and 3.5x times fewer FLOPs. Our model can perform infrared detection well across different infrared wavelengths. The optimal CSPNet configurations of our detection network selected by NAS show that the resulting architectures outperform the baseline.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Multi-spectral infrared object detection across different infrared wavelengths is a challenging task. Although some full-sized object detection models, such as YOLOv4 and ScaledYOLO, may achieve good infrared object detection, they are resource-demanding and unsuitable for real-time detection on edge devices. Tiny versions for object detection are proposed to meet the practical requirement, but they usually sacrifice model accuracy and generalization for efficiency. We propose an accurate and efficient object detector capable of performing real-time inference under the hardware constraints of an edge device by leveraging structural pruning, feature distillation, and neural architecture search (NAS). The experiments on FLIR and multi-spectral object detection datasets show that our model achieves comparable mAP to full-sized models while having 14x times fewer parameters and 3.5x times fewer FLOPs. Our model can perform infrared detection well across different infrared wavelengths. The optimal CSPNet configurations of our detection network selected by NAS show that the resulting architectures outperform the baseline.
2166.
Zhou, J.; Chen, F.; Lu, G.
Pruning Based Training-Free Neural Architecture Search Inproceedings
In: 2022 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, IEEE Computer Society, Los Alamitos, CA, USA, 2022.
@inproceedings{9859732,
title = {Pruning Based Training-Free Neural Architecture Search},
author = {J. Zhou and F. Chen and G. Lu},
url = {https://doi.ieeecomputersociety.org/10.1109/ICME52920.2022.9859732},
doi = {10.1109/ICME52920.2022.9859732},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
booktitle = {2022 IEEE International Conference on Multimedia and Expo (ICME)},
pages = {1-6},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Neural Architecture Search (NAS) plays an important role in searching for high-performance neural networks. How-ever, NAS algorithms are slow and require a terrific amount of computing resources, because they need to be trained on supernet or dense candidate networks to obtain information for evaluation. If the high-performance network architecture could be selected without training, it would eliminate a signif-icant part of the computational cost. Therefore, we propose a zero-cost metric called EX-score, which can represent the ex-pressivity of the network and rank the untrained architectures. To further reduce cost, we design a pruning based zero-cost neural architecture search framework (PZ-NAS) using EX-score. PZ-NAS can prune the initialised supernet rapidly and obtains hundreds of times faster speed performance, whilst archieving comparable accuracy property on CIFAR-IO and ImageNet.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) plays an important role in searching for high-performance neural networks. How-ever, NAS algorithms are slow and require a terrific amount of computing resources, because they need to be trained on supernet or dense candidate networks to obtain information for evaluation. If the high-performance network architecture could be selected without training, it would eliminate a signif-icant part of the computational cost. Therefore, we propose a zero-cost metric called EX-score, which can represent the ex-pressivity of the network and rank the untrained architectures. To further reduce cost, we design a pruning based zero-cost neural architecture search framework (PZ-NAS) using EX-score. PZ-NAS can prune the initialised supernet rapidly and obtains hundreds of times faster speed performance, whilst archieving comparable accuracy property on CIFAR-IO and ImageNet.
2165.
Malkova, Aleksandra
Reconstruction of Radio Power Maps with Deep Neural Networks in a Weakly Labeled Learning Context PhD Thesis
Universite Grenoble Alpes, 2022.
@phdthesis{malkova:tel-03740720,
title = {Reconstruction of Radio Power Maps with Deep Neural Networks in a Weakly Labeled Learning Context},
author = {Aleksandra Malkova},
url = {https://tel.archives-ouvertes.fr/tel-03740720},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
school = {Universite Grenoble Alpes},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2164.
Marchisio, Alberto; Mrazek, Vojtech; Massa, Andrea; Bussolino, Beatrice; Martina, Maurizio; Shafique, Muhammad
In: DyNN workshop at the 39th International Conference on Machine Learning, 2022.
@inproceedings{MarchisioDyNN2022,
title = {HARNAS: Neural Architecture Search Jointly Optimizing for Hardware Efficiency and Adversarial Robustness of Convolutional and Capsule Networks},
author = {Alberto Marchisio and Vojtech Mrazek and Andrea Massa and Beatrice Bussolino and Maurizio Martina and Muhammad Shafique},
url = {https://dynn-icml2022.github.io/papers/paper_18.pdf},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
booktitle = {DyNN workshop at the 39th International Conference on Machine Learning},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2163.
Sun, Zhenhong; Lin, Ming; Sun, Xiuyu; Tan, Zhiyu; Li, Hao; Jin, Rong
MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection Inproceedings
In: Chaudhuri, Kamalika; Jegelka, Stefanie; Song, Le; Szepesvari, Csaba; Niu, Gang; Sabato, Sivan (Ed.): Proceedings of the 39th International Conference on Machine Learning, pp. 20810–20826, PMLR, 2022.
@inproceedings{pmlr-v162-sun22cb,
title = {MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection},
author = {Zhenhong Sun and Ming Lin and Xiuyu Sun and Zhiyu Tan and Hao Li and Rong Jin},
editor = {Kamalika Chaudhuri and Stefanie Jegelka and Le Song and Csaba Szepesvari and Gang Niu and Sivan Sabato},
url = {https://proceedings.mlr.press/v162/sun22c.html},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
booktitle = {Proceedings of the 39th International Conference on Machine Learning},
volume = {162},
pages = {20810--20826},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {In object detection, the detection backbone consumes more than half of the overall inference cost. Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS). However, existing NAS methods for object detection require hundreds to thousands of GPU hours of searching, making them impractical in fast-paced research and development. In this work, we propose a novel zero-shot NAS method to address this issue. The proposed method, named MAE-DET, automatically designs efficient detection backbones via the Maximum Entropy Principle without training network parameters, reducing the architecture design cost to nearly zero yet delivering the state-of-the-art (SOTA) performance. Under the hood, MAE-DET maximizes the differential entropy of detection backbones, leading to a better feature extractor for object detection under the same computational budgets. After merely one GPU day of fully automatic design, MAE-DET innovates SOTA detection backbones on multiple detection benchmark datasets with little human intervention. Comparing to ResNet-50 backbone, MAE-DET is $+2.0%$ better in mAP when using the same amount of FLOPs/parameters, and is $1.54$ times faster on NVIDIA V100 at the same mAP. Code and pre-trained models are available here (https://github.com/alibaba/lightweight-neural-architecture-search).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In object detection, the detection backbone consumes more than half of the overall inference cost. Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS). However, existing NAS methods for object detection require hundreds to thousands of GPU hours of searching, making them impractical in fast-paced research and development. In this work, we propose a novel zero-shot NAS method to address this issue. The proposed method, named MAE-DET, automatically designs efficient detection backbones via the Maximum Entropy Principle without training network parameters, reducing the architecture design cost to nearly zero yet delivering the state-of-the-art (SOTA) performance. Under the hood, MAE-DET maximizes the differential entropy of detection backbones, leading to a better feature extractor for object detection under the same computational budgets. After merely one GPU day of fully automatic design, MAE-DET innovates SOTA detection backbones on multiple detection benchmark datasets with little human intervention. Comparing to ResNet-50 backbone, MAE-DET is $+2.0%$ better in mAP when using the same amount of FLOPs/parameters, and is $1.54$ times faster on NVIDIA V100 at the same mAP. Code and pre-trained models are available here (https://github.com/alibaba/lightweight-neural-architecture-search).
2162.
Sun, Zhenhong; Lin, Ming; Sun, Xiuyu; Tan, Zhiyu; Li, Hao; Jin, Rong
MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection Inproceedings
In: Chaudhuri, Kamalika; Jegelka, Stefanie; Song, Le; Szepesvari, Csaba; Niu, Gang; Sabato, Sivan (Ed.): Proceedings of the 39th International Conference on Machine Learning, pp. 20810–20826, PMLR, 2022.
@inproceedings{pmlr-v162-sun22c,
title = {MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection},
author = {Zhenhong Sun and Ming Lin and Xiuyu Sun and Zhiyu Tan and Hao Li and Rong Jin},
editor = {Kamalika Chaudhuri and Stefanie Jegelka and Le Song and Csaba Szepesvari and Gang Niu and Sivan Sabato},
url = {https://proceedings.mlr.press/v162/sun22c.html},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
booktitle = {Proceedings of the 39th International Conference on Machine Learning},
volume = {162},
pages = {20810--20826},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {In object detection, the detection backbone consumes more than half of the overall inference cost. Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS). However, existing NAS methods for object detection require hundreds to thousands of GPU hours of searching, making them impractical in fast-paced research and development. In this work, we propose a novel zero-shot NAS method to address this issue. The proposed method, named MAE-DET, automatically designs efficient detection backbones via the Maximum Entropy Principle without training network parameters, reducing the architecture design cost to nearly zero yet delivering the state-of-the-art (SOTA) performance. Under the hood, MAE-DET maximizes the differential entropy of detection backbones, leading to a better feature extractor for object detection under the same computational budgets. After merely one GPU day of fully automatic design, MAE-DET innovates SOTA detection backbones on multiple detection benchmark datasets with little human intervention. Comparing to ResNet-50 backbone, MAE-DET is $+2.0%$ better in mAP when using the same amount of FLOPs/parameters, and is $1.54$ times faster on NVIDIA V100 at the same mAP. Code and pre-trained models are available here (https://github.com/alibaba/lightweight-neural-architecture-search).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In object detection, the detection backbone consumes more than half of the overall inference cost. Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS). However, existing NAS methods for object detection require hundreds to thousands of GPU hours of searching, making them impractical in fast-paced research and development. In this work, we propose a novel zero-shot NAS method to address this issue. The proposed method, named MAE-DET, automatically designs efficient detection backbones via the Maximum Entropy Principle without training network parameters, reducing the architecture design cost to nearly zero yet delivering the state-of-the-art (SOTA) performance. Under the hood, MAE-DET maximizes the differential entropy of detection backbones, leading to a better feature extractor for object detection under the same computational budgets. After merely one GPU day of fully automatic design, MAE-DET innovates SOTA detection backbones on multiple detection benchmark datasets with little human intervention. Comparing to ResNet-50 backbone, MAE-DET is $+2.0%$ better in mAP when using the same amount of FLOPs/parameters, and is $1.54$ times faster on NVIDIA V100 at the same mAP. Code and pre-trained models are available here (https://github.com/alibaba/lightweight-neural-architecture-search).
2161.
Parallel and Distributed Methods for Autonomous Design of Artificial Neural Networks PhD Thesis
2022.
@phdthesis{GeorgeKyriakides,
title = {Parallel and Distributed Methods for Autonomous Design of Artificial Neural Networks},
url = {https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiXytvG4vX4AhXUXvEDHTLtD3wQFnoECAUQAQ&url=https%3A%2F%2Fdspace.lib.uom.gr%2Fbitstream%2F2159%2F27216%2F5%2FKyriakidesGeorgePhD2022.pdf&usg=AOvVaw3uanEdjCn9YtbbC59ScY2Z},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2160.
Wen, Long; Wang, You; Li, Xinyu
A new automatic convolutional neural network based on deep reinforcement learning for fault diagnosis Journal Article
In: Frontiers of Mechanical Engineering, vol. 17, 2022.
@article{WenFME2022,
title = {A new automatic convolutional neural network based on deep reinforcement learning for fault diagnosis},
author = { Long Wen and You Wang and Xinyu Li },
url = {https://link.springer.com/article/10.1007/s11465-022-0673-7},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
journal = {Frontiers of Mechanical Engineering},
volume = {17},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2159.
Fayyazifar, Najmeh
Deep learning and neural architecture search for cardiac arrhythmias classification PhD Thesis
2022.
@phdthesis{FayyazifarPhD,
title = {Deep learning and neural architecture search for cardiac arrhythmias classification},
author = {Najmeh Fayyazifar},
url = {https://ro.ecu.edu.au/theses/2553/},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}