mRMR Feature Selection

mRMR Feature Selection selects features for pattern classification by maximizing mutual information with the target while minimizing redundancy among features to produce compact, informative feature sets.

Key Features:

• Mutual Information-Based Selection: Uses mutual information to maximize statistical dependency between selected features and the target variable (maximal relevance) while reducing inter-feature redundancy (minimal redundancy).
• Minimal-Redundancy-Maximal-Relevance Criterion: Implements the minimal-redundancy-maximal-relevance criterion as the selection objective to balance relevance and redundancy.
• Incremental (First-Order) Feature Selection: Employs a first-order incremental selection scheme that evaluates each candidate feature's contribution according to the mRMR criterion.
• Two-Stage Algorithm with Wrapper Integration: Applies a two-stage workflow that first selects features using the mRMR criterion and subsequently refines the set using wrapper-based feature selectors.

Scientific Applications:

• Bioinformatics and Genomics: Applied to high-dimensional biological data in bioinformatics, genomics, and medical diagnostics to reduce dimensionality while retaining informative features.
• Classifier Performance Improvement: Used to improve classifiers such as naive Bayes, support vector machines (SVM), and linear discriminant analysis (LDA) by reducing redundancy and focusing on relevant features.
• Benchmark and Research Datasets: Validated on datasets including handwritten digits, arrhythmia data, NCI cancer cell lines, and lymphoma tissues.

Methodology:

Derives an equivalent form to the maximal dependency condition as the minimal-redundancy-maximal-relevance criterion, computes mutual information for relevance and redundancy, evaluates features incrementally using a first-order scheme, and applies a two-stage approach that can be followed by wrapper-based selectors.