BoltDB

BoltDB is an embedded key/value database written in Go. With shadow paging and MVCC by allowing only one writer and multiple readers, it guarantees ACID properties as well as fully serializable transactions. BoltDB stores key/value pairs in B+tree data storage, which allows fast read access. Since BoltDB only deals with a single file at a time, it performs better at recovery from a system crash.

History

In 2011, Howard Chu introduced MDB, a memory-mapped database backend for OpenLDAP, later renamed to [LMDB] (https://symas.com/lmdb/technical/)(Lightning Memory-Mapped Database). In 2013, BoltDB was initially started by Ben Johnson as a port of LMDB to Go, but then the two projects diverged as the author of Bolt decided to focus on simplicity and providing the easy-to-use Go API. The goal of BoltDB became to provide a simple, fast, and reliable database for projects that don't require a full database server such as Postgres or MySQL. With BoltDB being stable, its API fixed, and its file format fixed, the author considered the project a success. Leaving it in such a state, the project was abandoned by its author in 2017.

Checkpoints

Not Supported

Concurrency Control

Multi-version Concurrency Control (MVCC)

BoltDB supports lock-free MVCC using a single writer and multiple readers. It allows only one read-write transaction at a time but allows multiple read-only transactions at the same time. Allowing read-only transactions and read-write transactions depend on one another can cause deadlocks since if there exists a reader, another writer cannot periodically remap the file as needed by copy-on-write. Similarly, the co-occurrence of read-only transactions and read-write transactions in the same goroutine should also be avoided due to the same reason.

Data Model

Key/Value

Bolt is a key-value store that provides an ordered map, which allows easy access and lookup. All collections of key/value pairs are stored in buckets within which all keys must be unique. The keys are stored in byte-sorted order within a bucket. A bucket can be created with `Tx.CreateBucket` or `Tx.CreateBucketIfNotEXists` so that it can be created only if it doesn't exist.

Foreign Keys

Not Supported

Indexes

B+Tree

BoltDB stores key/value pairs in a B+ tree.

Isolation Levels

Serializable

BoltDB supports fully serializable ACID transactions. It allows only one read-write transaction at a time but allows as many read-only transactions as requested. When opening a database with `bolt.Open`, BoltDB obtains a file lock on the data file which prevents multiple processes from accessing the same database at the same time. A process will hang upon attempting to open an already open Bolt database until the process that opens the database closes it. BoltDB supports the ability to prevent an indefinite wait by passing a timeout option to the `Open` function.

Logging

Not Supported

Bolt saves data into a single memory-mapped file on disk. It doesn’t have a separate journal or write-ahead log.

Query Interface

Custom API

BoltDB supports basic key/value queries such as insert, delete and update with its own APIs. All `bucket` operations are organized and monitored by transactions `Tx`. For example, `Tx.CreateBucket` creates a bucket, and `Tx.DeleteBucket` removes a bucket. All basic key/value queries are performed on `bucket` using `Bucket.Put`, `Bucket.Get`, and `Bucket.Delete`. Since BoltDB stores its key in byte-sorted order within a bucket, it supports sequential iteration over a bucket with `Bucket.Cursor`. The `Cursor` allows moving to the first key (`First`), the last key (`Last`), the previous key (`Prev`), the next key (`Next`), and a specific key (`Seek`) in a bucket. Prefix scan is supported by combining `Seek` and `bytes.HasPrefix`. Range scan is supported by combing `Seek` and `bytes.Compare`. Besides `Bucket.Cursor`, `Bucket.ForEach` also supports iterating over all key/value pairs in a bucket.

Storage Architecture

Hybrid

There are only a few types in BoltDB: `DB`, `Bucket`, `Tx`, and `Cursor`. - The `DB` is a single file represented by pages on disk. Each page is commonly 4096 Bytes. The first two pages of the file store the metadata that keeps track of version, transaction id, and page size of the database, as well as the locations of the `freelist` and the first page id of data. The third page stores a `freelist`that keeps track the page id's of the free pages. The rest of the pages are the collection of buckets that store the key/value paris. - A `Bucket` is a collection of unique keys that are associated with values. Each bucket is represented using a B+ tree. When accessing a B+ tree, the nodes in the corresponding page are fetched into memory. The B+ tree used in BoltDB is different from common B+tree in the following aspects: - While B+ trees typically have n+1 values and n keys in each node, the number of values and the number of keys are equal in BoltDB B+ tree. - In BoltDB B+ tree, the value field in a non-leaf node stores the page id of its child node and the corresponding key stores the first key in the child node. - There are no pointers between sibling leaf nodes in BoltDB B+ tree.

Storage Model

Custom

BoltDB uses key/value storage, which does not involve rows or columns.

Storage Organization

Copy-on-Write / Shadow Paging

BoltDB saves data into a single memory-mapped file on disk. Write-ahead log is not necessary since BoltDB only deals with one file at a time. With copy-on-write, when writing to a page, BoltDB makes updates on the copy of the original page and updates the pointer to point at the new page upon commit.

System Architecture

Embedded

Views

Virtual Views

Bolt allows only one read-write transaction at a time but allows as many read-only transactions as requested. The view obtained by each transaction is consistent with the data present at the beginning of the transaction. BoltDB recommends three types of transactions: read-write transactions with `DB.Update`, read-only transactions with `DB.View` and batch read-write transactions with `DB.Batch`. Along with the three recommended transactions, BoltDB also allows manual transaction management with `DB.Begin`.

Embeddings

People Also Viewed

BoltDB Logo
Website

https://github.com/boltdb/bolt

Source Code

https://github.com/boltdb/bolt

Tech Docs

https://godoc.org/github.com/boltdb/bolt

Developer

Ben Johnson

Country of Origin

US

Start Year

2013

End Year

2017

Project Type

Open Source

Written in

Go

Supported languages

Go

Inspired By

LMDB

Operating Systems

Android, BSD, iOS, Linux, OS X, Solaris, Windows

Licenses

MIT

Embeddings

People Also Viewed