RelativeNAS

RelativeNAS: Neural Architecture Search for Convolutional Neural Networks

RelativeNAS automates convolutional neural network (CNN) design using a neural architecture search (NAS) framework that optimizes candidate architectures through pairwise learning and low-fidelity performance estimation.

Key Features:

• Joint Learning Mechanism: Pairs fast learners (decoded networks with lower loss values) and slow learners to improve search efficiency through relative performance comparison.
• Low-Fidelity Performance Estimation: Uses low-fidelity evaluation to distinguish fast and slow learners, reducing computational cost during candidate architecture training.
• Benchmark Performance: Achieves 24.88% top-1 error on ImageNet, outperforming DARTS and AmoebaNet-B by 1.82% and 1.12%, respectively.
• Computational Efficiency: Discovers optimal network cells in nine hours using a single NVIDIA Tesla V100 GPU, 3.75× faster than DARTS and 7875× faster than AmoebaNet.
• Transferability: Transfers architectures discovered on CIFAR-10 to object detection, semantic segmentation, and keypoint detection, achieving 73.1% mAP on PASCAL VOC, 78.7% mIoU on Cityscapes, and 68.5% AP on MSCOCO.

Scientific Applications:

• Image Analysis and Pattern Recognition: Supports automated design of high-performance CNNs for medical imaging, object detection, semantic segmentation, and keypoint detection.

Methodology:

Combines differentiable NAS and population-based NAS using a pairwise learning strategy between fast and slow learners, guided by low-fidelity performance estimation to accelerate and optimize architecture search.