Compression

Dictionary Encoding

String fields in Cubrick are dictionary encoded, for both dimensions (i.e., indices) and metrics (i.e., values). Internally, Cubrick processes string fields using their encoded integers, and only converts them back when returning the results to the users.

Cubrick also uses BESS (Bit-Encoded Sparse Structure) encoding for compressing the multidimensional index for each cell (i.e., a group of metrics corresponding to the same dimension).

Indexes

Hash Table

Data Model

Column Family / Wide-Column

Cubrick stores data in bricks (i.e., partitions) in a column-wise fashion. In each brick, each column has a dynamic vector to store the metrics or the BESS encoded indices. Cells in a brick are unordered, and they are only appended to the end of the brick in the data ingestion.