DeepAction

DeepAction automates classification of animal behavior in video to enable quantitative behavioral analysis in ethology, neuroscience, and psychology.

Key Features:

• MATLAB implementation: Implemented as a MATLAB toolbox for processing and analyzing video-based behavior data.
• Deep learning framework: Two-tier neural network architecture using features extracted from raw video frames by a pretrained convolutional neural network (CNN) and classified by a recurrent neural network (RNN) for temporal sequence modeling.
• High accuracy with minimal training data: Evaluated on benchmark datasets from rodents and octopuses and reported to achieve high classification accuracy with limited training samples.
• Performance benchmarking: Compared against human annotators and other contemporary techniques, matching or surpassing human agreement levels in behavior annotation tasks.
• Confidence scoring mechanism: Generates confidence scores for each classification output to estimate classifier reliability.

Scientific Applications:

• Ethology: Automated annotation of animal behaviors in video to support descriptive and quantitative studies of species-specific behavior.
• Neuroscience: Quantification of behavioral phenotypes in neuroscience experiments using rodents and cephalopods.
• Psychology: Objective behavioral scoring in psychological experiments involving animal models.

Methodology:

Video frames are preprocessed and features are extracted with a pretrained convolutional neural network, and these features are input to a recurrent neural network classifier trained to recognize and categorize specific behaviors.