Vision Permutator

Vision Permutator encodes 2D visual feature maps by independently projecting height and width dimensions to capture long-range spatial dependencies for image recognition.

Key Features:

• Separate encoding of spatial dimensions: Processes height and width dimensions independently using linear projections to retain 2D positional information.
• Long-range dependency capture without attention: Encodes features across separate dimensions to capture long-range dependencies without relying on attention mechanisms.
• Mutually complementing aggregation: Aggregates outputs from the separate encodings in a complementary manner to enhance representation expressiveness.
• Avoidance of spatial convolutions: Operates without spatial convolutions, implementing an MLP-like architecture for visual recognition.

Scientific Applications:

• ImageNet classification (25M): Achieves 81.5% top-1 accuracy on ImageNet using 25 million learnable parameters.
• ImageNet classification (88M): Achieves 83.2% top-1 accuracy on ImageNet using 88 million parameters, outperforming recent MLP-like networks.
• Alternative to CNNs and vision transformers: Serves as a competitive architecture to convolutional neural networks and vision transformers at similar model sizes, delivering high accuracy with fewer parameters.

Methodology:

Apply linear projections separately to height and width dimensions of 2D feature representations, aggregate the separate outputs via mutually complementing aggregation, and avoid both spatial convolutions and attention mechanisms; this reduces computational complexity and attains high accuracy on ImageNet without requiring ImageNet-22k.