SSD-KD

SSD-KD performs self-supervised diverse knowledge distillation to produce lightweight MobileNetV2-based models for multi-disease skin lesion classification from dermoscopic images.

Key Features:

• Knowledge Distillation Framework: Integrates diverse forms of knowledge into a unified knowledge distillation (KD) framework to improve student model performance.
• Intra-Instance Relational Feature Representation: Captures complex relationships within individual instances via an intra-instance relational feature representation.
• Dual Relational Knowledge Distillation Architecture: Employs a dual relational KD architecture trained self-supervised that transfers intra- and inter-instance relational knowledge and uses weighted softened outputs.
• Lightweight Model Efficiency: Uses MobileNetV2 as the student backbone to achieve high classification accuracy with minimal parameters and computation.

Scientific Applications:

• Multi-disease skin lesion classification: Classifies multiple skin diseases from dermoscopic images for dermatological diagnosis tasks.
• Benchmarking on ISIC 2019: Achieved up to 85% accuracy on eight-class skin disease classification using MobileNetV2 on the ISIC 2019 dataset.
• Ablation study validation: Ablation experiments confirmed the contribution of intra- and inter-instance relational knowledge integration to performance improvements.

Methodology:

Self-supervised training of a dual relational knowledge distillation architecture that transfers knowledge from a complex teacher model to a lightweight student (MobileNetV2) by integrating intra-instance and inter-instance relational feature representations and using weighted softened outputs, validated on the ISIC 2019 dermoscopic image dataset.