Sparkey is a persistent key-value storage library designed for read-heavy workloads with infrequent bulk inserts. It operates as an immutable on-disk hash table, providing a C API and command-line utilities to manage index and log files. The system supports multiple concurrent readers but restricts writes to a single writer per storage unit, optimizing for fast random access reads and efficient bulk data loading.
- Source Code
- https://github.com/spotify/sparkey[01]
- Developer
- Country of Origin
- SE
- Start Year
- 2013
- End Year
- 2023
- Project Type
- Open Source
- Written in
- C
- Inspired By
- cdb
- Operating System
- Linux
- License
- Apache v2
It provides a foundation for building read-optimized services by combining fast bulk write capabilities with low-latency random access reads. The project includes official bindings for Python and Java, along with community contributions for other languages.
Sparkey is a persistent key-value storage library designed for read-heavy workloads with infrequent bulk inserts. It operates as an immutable on-disk hash table, providing a C API and command-line utilities to manage index and log files. The system supports multiple concurrent readers but restricts writes to a single writer per storage unit, optimizing for fast random access reads and efficient bulk data loading.
It provides a foundation for building read-optimized services by combining fast bulk write capabilities with low-latency random access reads. The project includes official bindings for Python and Java, along with community contributions for other languages.
History
Developed internally at Spotify to serve rarely updated data efficiently, Sparkey was open-sourced in 2013.
The system's source code repository has not been updated since 2023 and is now considered abandoned.