SongExplorer

SongExplorer segments animal acoustic communication signals, including airborne sounds and substrate-borne vibrations, using deep learning to support discovery, annotation, training, and automated segmentation for bioacoustic research.

Key Features:

• Discovery from raw data: Facilitates identification of song events and novel acoustic types from raw recordings.
• Manual annotation: Supports manual labeling of song events to provide ground truth for model training.
• Supervised training (deep CNN): Employs a deep convolutional neural network trained on annotated data to learn song-event classes.
• Automated segmentation: Uses the trained model to segment recordings into distinct acoustic events automatically.
• Audio–video synchronization: Enables correlation of acoustic events with synchronized video to link sounds to visual behaviors.
• Low-dimensional visualization: Provides low-dimensional embeddings to visualize large numbers of song events and detect mislabeled examples.
• Performance and accuracy: Demonstrates higher accuracy than heuristic algorithms and achieves segmentation accuracy comparable to two expert human annotators.

Scientific Applications:

• Novel song-type discovery: Enables rapid detection of novel song types within new or previously studied species.
• Behavioral analysis: Supports linking acoustic signals to visual behaviors for studies of communication and courtship.
• Biodiversity and ecological monitoring: Facilitates detection and quantification of species-specific acoustic signatures for monitoring.
• Evolution and conservation studies: Provides segmented, labeled acoustic data for investigations of signal evolution and conservation assessments.

Methodology:

Manual annotation of song events; supervised training of a deep convolutional neural network on annotated data; automated segmentation using the trained CNN; generation of low-dimensional visualizations of song-event embeddings; synchronization of audio with video for cross-modal correlation.