SLEAP

SLEAP performs multi-animal pose estimation and instance tracking using deep learning to quantify body-part positions for behavioral and neuroscience studies.

Key Features:

• Multi-Animal Pose Estimation: Performs simultaneous body-part localization for multiple animals during social interactions.
• Instance Tracking: Links detected body parts across frames to maintain animal identities over time.
• Configurable Neural Network Architectures: Provides customizable neural network architectures to adapt models to dataset-specific requirements.
• Inference Techniques and Tracking Algorithms: Implements multiple inference methods and tracking algorithms that can be fine-tuned for transitioning from single-animal to multi-animal estimation.
• Accuracy and Performance: Reports less than 2.8 pixels error on 95% of points and can process full-size frames (1024 × 1024 pixels) at up to 320 frames per second.
• Implementation: Implemented in Python.

Scientific Applications:

• Behavioral Analysis: Enables quantitative analysis of animal behavior by providing precise body-part trajectories.
• Social Interaction Studies: Supports studies of social interactions by tracking multiple animals simultaneously.
• Neuroscience and Ethology: Facilitates investigations into how brains generate and pattern behaviors across species such as flies, bees, and mice.

Methodology:

Uses deep learning–based pose estimation with configurable neural network architectures, multiple inference methods and tracking algorithms for instance tracking, and model training and evaluation (reported as pixel-error and frame-rate benchmarks); implemented in Python.