CLR-RNF

CLR-RNF performs filter-level pruning of convolutional neural networks using Cross-Layer Ranking and k-Reciprocal Nearest Filter selection to reduce floating-point operations (FLOPs) and parameters while preserving classification accuracy.

Key Features:

• Long-Tail Pruning Problem Identification: Identifies the "long-tail" issue in magnitude-based pruning where less significant weights are disproportionately pruned.
• Computation-Aware Weight Importance Measurement: Computes a computation-aware metric to assess the importance of individual weights for pruning decisions.
• Cross-Layer Ranking (CLR): Ranks weights across different layers by computed importance and identifies bottom-ranked weights for structured per-layer sparsity reduction.
• Recommendation-Based Filter Selection: Each filter recommends a group of its closest filters to inform which connections to maintain during pruning.
• k-Reciprocal Nearest Filter (RNF) Selection Scheme: Preserves filters based on their presence in the intersection of recommended groups using a k-reciprocal nearest-filter criterion to select mutually relevant filters.
• Nonlearning Pruned-Structure and Filter Selection: Determines both the pruned network structure and filter selection via nonlearning procedures rather than additional learning-based optimization.

Scientific Applications:

• Image classification: Applied to image classification benchmarks to reduce computational cost while maintaining or minimally affecting accuracy.
• CIFAR-10 results: On VGGNet-16 CLR-RNF reduces FLOPs by 74.1% and parameters by 95.0% while improving accuracy by 0.3%.
• ImageNet results: On ResNet-50 CLR-RNF reduces FLOPs by 70.2% and parameters by 64.8% with a top-five accuracy drop of 1.7%.

Methodology:

Identify the long-tail pruning problem, compute a computation-aware weight importance metric, apply Cross-Layer Ranking to order weights, use per-filter recommendation groups, and select preserved filters via a k-reciprocal nearest-filter intersection for pruning; both structure determination and filter selection are nonlearning processes.