HULK

HULK (Histosketching Using Little K-mers) is a software tool to address the challenges posed by the exponential growth in microbiome data. It introduces a method for compactly representing microbiome sequencing data by creating similarity-preserving sketches of streaming k-mer spectra, known as histosketches. These histosketches serve as dimensionality reduction, enabling efficient representation, dissimilarity estimation, rapid microbiome catalogue searching, and classification of microbiome samples in near real-time.

The core functionality of HULK involves applying streaming histogram sketching to microbiome samples. This process compresses the k-mer spectra of a microbiome into a histosketch, significantly reducing the data's complexity while preserving its ability to be analyzed for similarity. Using pairwise Jaccard similarity estimation, HULK can cluster histosketches by sample type, facilitating rapid searches for microbiome similarity through a locality-sensitive hashing indexing scheme.

A key application demonstrated by HULK's developers involves using histosketches to train machine learning classifiers. For instance, a random forest classifier was trained using 108 novel microbiome samples from premature neonates. This classifier achieved high accuracy (97%) and precision (96%) in predicting antibiotic treatment among the neonates, and it could classify microbiome data streams in less than 3 seconds.