AutoML | Literature on Neural Architecture Search

1276.

Zhang, Yongan; Fu, Yonggan; Jiang, Weiwen; Li, Chaojian; You, Haoran; Li, Meng; Chandra, Vikas; Lin, Yingyan

DIAN: Differentiable Accelerator-Network Co-Search Towards Maximal DNN Efficiency Proceedings Article

In: 2021 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), pp. 1-6, 2021.

Links | BibTeX

1275.

Repin, Denis; Petrov, Tatjana

Automated Deep Abstractions for Stochastic Chemical Reaction Networks Journal Article

In: Information and Computation, pp. 104788, 2021, ISSN: 0890-5401.

Abstract | Links | BibTeX

@article{REPIN2021104788,

title = {Automated Deep Abstractions for Stochastic Chemical Reaction Networks},

author = {Denis Repin and Tatjana Petrov},

url = {https://www.sciencedirect.com/science/article/pii/S0890540121001048},

doi = {https://doi.org/10.1016/j.ic.2021.104788},

issn = {0890-5401},

year  = {2021},

date = {2021-01-01},

journal = {Information and Computation},

pages = {104788},

abstract = {Predicting stochastic cellular dynamics as emerging from the mechanistic models of molecular interactions is a long-standing challenge in systems biology: low-level chemical reaction network (CRN) models give rise to a highly-dimensional continuous-time Markov chain (CTMC) which is computationally demanding and often prohibitive to analyse in practice. A recently proposed abstraction method uses deep learning to replace this CTMC with a discrete-time continuous-space process, by training a mixture density deep neural network with traces sampled at regular time intervals (which can be obtained either by simulating a given CRN or as time-series data from experiment). The major advantage of such abstraction is that it produces a computational model that is dramatically cheaper to execute, while it preserves the statistical features of the training data. In general, the abstraction accuracy improves with the amount of training data. However, depending on the CRN, the overall quality of the method – the efficiency gain and abstraction accuracy – will also depend on the choice of neural network architecture given by hyper-parameters such as the layer types and connections between them. As a consequence, in practice, the modeller has to take care of finding the suitable architecture manually, for each given CRN, through a tedious and time-consuming trial-and-error cycle. In this paper, we propose to further automatise deep abstractions for stochastic CRNs, through learning the neural network architecture along with learning the transition kernel of the abstract process. Automated search of the architecture makes the method applicable directly to any given CRN, which is time-saving for deep learning experts and crucial for non-specialists. We implement the method and demonstrate its performance on a number of representative CRNs with multi-modal emergent phenotypes. Moreover, we showcase that deep abstractions can be used for efficient multi-scale simulations, which are otherwise computationally intractable. To this end, we define a scenario where multiple CRN instances interact across a spatial grid via shared species. Finally, we discuss the limitations and challenges arising when using deep abstractions.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

Predicting stochastic cellular dynamics as emerging from the mechanistic models of molecular interactions is a long-standing challenge in systems biology: low-level chemical reaction network (CRN) models give rise to a highly-dimensional continuous-time Markov chain (CTMC) which is computationally demanding and often prohibitive to analyse in practice. A recently proposed abstraction method uses deep learning to replace this CTMC with a discrete-time continuous-space process, by training a mixture density deep neural network with traces sampled at regular time intervals (which can be obtained either by simulating a given CRN or as time-series data from experiment). The major advantage of such abstraction is that it produces a computational model that is dramatically cheaper to execute, while it preserves the statistical features of the training data. In general, the abstraction accuracy improves with the amount of training data. However, depending on the CRN, the overall quality of the method – the efficiency gain and abstraction accuracy – will also depend on the choice of neural network architecture given by hyper-parameters such as the layer types and connections between them. As a consequence, in practice, the modeller has to take care of finding the suitable architecture manually, for each given CRN, through a tedious and time-consuming trial-and-error cycle. In this paper, we propose to further automatise deep abstractions for stochastic CRNs, through learning the neural network architecture along with learning the transition kernel of the abstract process. Automated search of the architecture makes the method applicable directly to any given CRN, which is time-saving for deep learning experts and crucial for non-specialists. We implement the method and demonstrate its performance on a number of representative CRNs with multi-modal emergent phenotypes. Moreover, we showcase that deep abstractions can be used for efficient multi-scale simulations, which are otherwise computationally intractable. To this end, we define a scenario where multiple CRN instances interact across a spatial grid via shared species. Finally, we discuss the limitations and challenges arising when using deep abstractions.

Close

1274.

Yang, Shangshang; Tian, Ye; Xiang, Xiaoshu; Peng, Shichen; Zhang, Xingyi

Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation Technical Report

2021.

Links | BibTeX

1273.

Xie, Xiangning; Liu, Yuqiao; Sun, Yanan; Yen, Gary G; Xue, Bing; Zhang, Mengjie

BenchENAS: A Benchmarking Platform for Evolutionary Neural Architecture Search Technical Report

2021.

Links | BibTeX

1272.

Zhou, Xun; Qin, A K; Sun, Yanan; Tan, Kay Chen

A Survey of Advances in Evolutionary Neural Architecture Search Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 950-957, 2021.

Links | BibTeX

1271.

Shi, Rui; Luo, Jianping; Liu, Qiqi

Fast Evolutionary Neural Architecture Search Based on Bayesian Surrogate Model Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 1217-1224, 2021.

Links | BibTeX

1270.

Cai, Ronghong; Luo, Jianping

Multi-Task Learning for Multi-Objective Evolutionary Neural Architecture Search Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 1680-1687, 2021.

Links | BibTeX

1269.

Namekawa, Shizuma; Tezuka, Taro

Evolutionary Neural Architecture Search by Mutual Information Analysis Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 966-972, 2021.

Links | BibTeX

1268.

Zhao, Liang; Fang, Wei

An Efficient and Flexible Automatic Search Algorithm for Convolution Network Architectures Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 2203-2210, 2021.

Links | BibTeX

1267.

Zhang, Yuge; Yan, Chenqian; Zhang, Quanlu; Zhang, Li Lyna; Yang, Yaming; Gao, Xiaotian; Yang, Yuqing

AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing Technical Report

2021.

Links | BibTeX

1266.

Huang, Junhao; Xue, Bing; Sun, Yanan; Zhang, Mengjie

A Flexible Variable-length Particle Swarm Optimization Approach to Convolutional Neural Network Architecture Design Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 934-941, 2021.

Links | BibTeX

1265.

Andersen, Hayden; Stevenson, Sean; Ha, Tuan; Gao, Xiaoying; Xue, Bing

Evolving Neural Networks for Text Classification using Genetic Algorithm-based Approaches Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 1241-1248, 2021.

Links | BibTeX

1264.

Vieira, Carlos; Cáceres, Leslie Pérez; Bezerra, Leonardo C T

Evaluating Anytime Performance on NAS-Bench-101 Proceedings Article

In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp. 1249-1256, 2021.

Links | BibTeX

1263.

Xu, Hang; Kang, Ning; Zhang, Gengwei; Xie, Chuanlong; Liang, Xiaodan; Li, Zhenguo

NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models Technical Report

2021.

Links | BibTeX

1262.

Shim, Jae-hun; Kang, Suk-ju

Neural Architecture Search for Light-weight Multi-touch Classification Proceedings Article

In: 2021 36th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), pp. 1-3, 2021.

Links | BibTeX

1261.

Ding, Yuhui; Yao, Quanming; Zhao, Huan; Zhang, Tong

DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks Proceedings Article

In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 279–288, Association for Computing Machinery, Virtual Event, Singapore, 2021, ISBN: 9781450383325.

Abstract | Links | BibTeX

@inproceedings{10.1145/3447548.3467447,

title = {DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks},

author = {Yuhui Ding and Quanming Yao and Huan Zhao and Tong Zhang},

url = {https://doi.org/10.1145/3447548.3467447},

doi = {10.1145/3447548.3467447},

isbn = {9781450383325},

year  = {2021},

date = {2021-01-01},

urldate = {2021-01-01},

booktitle = {Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining},

pages = {279–288},

publisher = {Association for Computing Machinery},

address = {Virtual Event, Singapore},

series = {KDD '21},

abstract = {In this paper, we propose a novel framework to automatically utilize task-dependent 

semantic information which is encoded in heterogeneous information networks (HINs). 

Specifically, we search for a meta graph, which can capture more complex semantic 

relations than a meta path, to determine how graph neural networks (GNNs) propagate 

messages along different types of edges. We formalize the problem within the framework 

of neural architecture search (NAS) and then perform the search in a differentiable 

manner. We design an expressive search space in the form of a directed acyclic graph 

(DAG) to represent candidate meta graphs for a HIN, and we propose task-dependent 

type constraint to filter out those edge types along which message passing has no 

effect on the representations of nodes that are related to the downstream task. The 

size of the search space we define is huge, so we further propose a novel and efficient 

search algorithm to make the total search cost on a par with training a single GNN 

once. Compared with existing popular NAS algorithms, our proposed search algorithm 

improves the search efficiency. We conduct extensive experiments on different HINs 

and downstream tasks to evaluate our method, and experimental results show that our 

method can outperform state-of-the-art heterogeneous GNNs and also improves efficiency 

compared with those methods which can implicitly learn meta paths.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

Close

1260.

Li, Jihao; Weinmann, Martin; Sun, Xian; Diao, Wenhui; Feng, Yingchao; Fu, Kun

Random Topology and Random Multiscale Mapping: An Automated Design of Multiscale and Lightweight Neural Network for Remote-Sensing Image Recognition Journal Article

In: IEEE Transactions on Geoscience and Remote Sensing, pp. 1-17, 2021.

Links | BibTeX

1259.

Wang, Xin; Zhu, Wenwu

Automated Machine Learning on Graph Proceedings Article

In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 4082–4083, Association for Computing Machinery, Virtual Event, Singapore, 2021, ISBN: 9781450383325.

Abstract | Links | BibTeX

1258.

Geiping, Jonas; Lukasik, Jovita; Keuper, Margret; Moeller, Michael

DARTS for Inverse Problems: a Study on Hyperparameter Sensitivity Journal Article

In: CoRR, vol. abs/2108.05647, 2021.

Links | BibTeX

1257.

Razali, Mohd Norhisham; Moung, Ervin Gubin; Yahya, Farashazillah; Hou, Chong Joon; Hanapi, Rozita; Mohamed, Raihani; Hashem, Ibrahim Abakr Targio

Indigenous Food Recognition Model Based on Various Convolutional Neural Network Architectures for Gastronomic Tourism Business Analytics Journal Article

In: Inf., vol. 12, no. 8, pp. 322, 2021.

Links | BibTeX

1256.

Chen, Boyu; Li, Peixia; Li, Baopu; Lin, Chen; Li, Chuming; Sun, Ming; Yan, Junjie; Ouyang, Wanli

BN-NAS: Neural Architecture Search with Batch Normalization Technical Report

2021.

Links | BibTeX

1255.

Benmeziane, Hadjer; Maghraoui, Kaoutar El; Ouarnoughi, Hamza; Niar, Smaïl; Wistuba, Martin; Wang, Naigang

Hardware-Aware Neural Architecture Search: Survey and Taxonomy Proceedings Article

In: Zhou, Zhi-Hua (Ed.): Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 4322–4329, ijcai.org, 2021.

Links | BibTeX

1254.

Li, Yanxi; Yang, Zhaohui; Wang, Yunhe; Xu, Chang

Neural Architecture Dilation for Adversarial Robustness Technical Report

2021.

Links | BibTeX

1253.

Hosseini, Mahdi S.; Zhang, Jia Shu; Liu, Zhe; Fu, Andre; Su, Jingxuan; Tuli, Mathieu; Plataniotis, Konstantinos N.

CONet: Channel Optimization for Convolutional Neural Networks Technical Report

2021.

Links | BibTeX

1252.

Zhao, Changsheng; Hua, Ting; Shen, Yilin; Lou, Qian; Jin, Hongxia

Automatic Mixed-Precision Quantization Search of BERT Proceedings Article

In: Zhou, Zhi-Hua (Ed.): Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 3427–3433, ijcai.org, 2021.

Links | BibTeX

1251.

Tang, Sheyang; Hosseini, Mahdi S.; Chen, Lina; Varma, Sonal; Rowsell, Corwyn; Damaskinos, Savvas; Plataniotis, Konstantinos N.; Wang, Zhou

Probeable DARTS with Application to Computational Pathology Technical Report

2021.

Links | BibTeX

1250.

Zhao, Pengyu; Xiao, Kecheng; Zhang, Yuanxing; Bian, Kaigui; Yan, Wei

AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System Proceedings Article

In: Zhou, Zhi-Hua (Ed.): Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 2104–2110, ijcai.org, 2021.

Links | BibTeX

1249.

Lyu, Bo; Yang, Yin; Wen, Shiping; Huang, Tingwen; Li, Ke

Neural Architecture Search for Portrait Parsing Journal Article

In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-10, 2021.

Links | BibTeX

1248.

Liu, Chia-Hsiang; Han, Yu-Shin; Sung, Yuan-Yao; Lee, Yi; Chiang, Hung-Yueh; Wu, Kai-Chiang

FOX-NAS: Fast, On-device and Explainable Neural Architecture Search Technical Report

2021.

Links | BibTeX

1247.

Kim, Youngkee; Yun, Won Joon; Lee, Youn Kyu; Jung, Soyi; Kim, Joongheon

Trends in Neural Architecture Search: Towards the Acceleration of Search Technical Report

2021.

Links | BibTeX

1246.

Ganepola, Vayangi Vishmi Vishara; Wirasingha, Torin

Generative Adversarial Networks Using Neural Architecture Search for Semantic Image Segmentation Proceedings Article

In: 2021 IEEE 4th International Conference on Big Data and Artificial Intelligence (BDAI), pp. 236-241, 2021.

Links | BibTeX

1245.

Wang, Ruochen; Chen, Xiangning; Cheng, Minhao; Tang, Xiaocheng; Hsieh, Cho-Jui

RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving Proceedings Article

In: ICCV 2021, 2021.

Links | BibTeX

1244.

Hou, Pengfei; Jin, Ying; Chen, Yukang

Single-DARTS: Towards Stable Architecture Search Technical Report

2021.

Links | BibTeX

1243.

Fan, Yi; Zhou, Quoqiang; Zhang, Weifeng; Bao, Shudi; Shen, Jun

Determining learning direction via multi-controller model for stably searching generative adversarial networks Journal Article

In: Neurocomputing, vol. 464, pp. 37-47, 2021, ISSN: 0925-2312.

Abstract | Links | BibTeX

1242.

Peng, Jiefeng; Zhang, Jiqi; Li, Changlin; Wang, Guangrun; Liang, Xiaodan; Lin, Liang

Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift Technical Report

2021.

Links | BibTeX

1241.

Calisto, Maria Baldeon; Lai-Yuen, Susana K.

EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation Journal Article

In: Artificial Intelligence in Medicine, vol. 119, pp. 102154, 2021, ISSN: 0933-3657.

Abstract | Links | BibTeX

@article{BALDEONCALISTO2021102154,

title = {EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation},

author = {Maria Baldeon Calisto and Susana K. Lai-Yuen},

url = {https://www.sciencedirect.com/science/article/pii/S0933365721001470},

doi = {https://doi.org/10.1016/j.artmed.2021.102154},

issn = {0933-3657},

year  = {2021},

date = {2021-01-01},

urldate = {2021-01-01},

journal = {Artificial Intelligence in Medicine},

volume = {119},

pages = {102154},

abstract = {Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.

Close

1240.

Boutros, Fadi; Siebke, Patrick; Klemt, Marcel; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation Technical Report

2021.

Links | BibTeX

1239.

Chitty-Venkata, Krishna Teja; Somani, Arun K.

Array-Aware Neural Architecture Search Proceedings Article

In: 2021 IEEE 32nd International Conference on Application-specific Systems, Architectures and Processors (ASAP), pp. 125-132, 2021.

Links | BibTeX

1238.

Zhan, Zheng; Gong, Yifan; Zhao, Pu; Yuan, Geng; Niu, Wei; Wu, Yushu; Zhang, Tianyun; Jayaweera, Malith; Kaeli, David R.; Ren, Bin; Lin, Xue; Wang, Yanzhi

Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search Technical Report

2021.

Links | BibTeX