DeepSpectrumLite

DeepSpectrumLite applies transfer learning with pre-trained image convolutional neural networks to enable lightweight on-device audio classification by converting raw audio into Mel spectrograms for DNN training and inference.

Key Features:

• Transfer Learning: Leverages pre-trained image CNNs to adapt visual feature extractors for audio classification tasks.
• Mel Spectrogram Generation and Augmentation: Dynamically generates and augments Mel spectrogram plots from raw audio signals for model input.
• CNN Fine-Tuning: Fine-tunes specific convolutional neural network architectures on spectrogram representations for classification.
• Lightweight DNNs for On-Device Inference: Targets reduced trainable parameters and computational complexity to enable on-device, real-time inference.
• Measured Inference Performance: Demonstrated mean inference lag of 242.0 milliseconds using DenseNet121 on a Motorola moto e7 plus smartphone.

Scientific Applications:

• Paralinguistic Analysis: Classification of non-linguistic vocal attributes from speech audio.
• Emotion Recognition: Detection of emotional states from speech and music audio recordings.
• Social Signal Processing: Analysis of social and communicative signals from audio data.
• COVID-19 Cough Detection: Classification of cough sounds for COVID-19 detection studies.
• COVID-19 Speech Analysis: Analysis of speech signals for COVID-19 related biomarkers.
• Snore Sound Classification: Classification of snore recordings for sleep-related audio analysis.

Methodology:

Dynamic generation and augmentation of Mel spectrogram plots from raw audio; transfer learning using pre-trained image CNNs and fine-tuning of CNN architectures on spectrogram inputs; on-device real-time inference benchmarking (e.g., DenseNet121 mean lag 242.0 ms on Motorola moto e7 plus).