scikit-dimension

scikit-dimension estimates intrinsic dimensionality of datasets to quantify data complexity for machine learning, dimensionality reduction, and related analyses.

Key Features:

• Uniform Implementation: Provides a consistent framework implementing multiple intrinsic dimension (ID) estimators for both global and local estimation.
• Integration with scikit-learn: Exposes estimators via the scikit-learn API to integrate with scikit-learn-based workflows.
• Synthetic Data Generation: Includes generators for synthetic toy and benchmark datasets commonly used in the literature.
• Comprehensive Benchmarking: Contains benchmarking evaluations across over 500 datasets, including both real-life and synthetic datasets.
• Quality Assurance Tools: Incorporates code quality measures such as test coverage, unit testing, and continuous integration.

Scientific Applications:

• Data Science: Facilitates dimensionality reduction by providing estimates of the true complexity of datasets.
• Machine Learning Research: Enables benchmarking and comparison of different intrinsic dimension estimation methods.
• Complex Systems Analysis: Assists analysis of high-dimensional data from complex systems by revealing intrinsic dimensional structure.

Methodology:

The package implements a diverse array of intrinsic dimension estimators covering both global and local approaches and exposes them via the scikit-learn API.