DBDB.io The Encyclopedia of Database Systems · Est. 2017
Database of Databases

Database Entry

Bedrock


Bedrock is a distributed relational DBMS built on top of SQLite. It is a distributed relational database management system designed for geo-replication. It was initially made for (and is owned by) Expensify. Its storage is based on a distributed ledger.[04]

Source Code
https://github.com/Expensify/Bedrock[02]
Country of Origin
US
Start Year
2016 [25]
Coding Agents
Project Type
Open Source
Written in
C++
Supported Languages
C++
Embeds / Uses
SQLite
Inspired By
SQLite
Compatible With
MySQL
Operating Systems
Linux, macOS
License
LGPL v3

Database Entry

Bedrock


Bedrock is a distributed relational DBMS built on top of SQLite. It is a distributed relational database management system designed for geo-replication. It was initially made for (and is owned by) Expensify. Its storage is based on a distributed ledger.[04]

History[03][05]


BedRock is the system that backs Expensify, the expense management company. It had been used for eight years prior to being launched. It was originally created as an in house solution to the strict database constraints of financial institutions - response time within milliseconds, transaction logging and authentication, and replication of multiple servers.

Checkpoints[02][06][07]


Same as SQLite

Bedrock inherits checkpoint support from SQLite, hence supports blocking and non-blocking checkpoints.

Compression[08][09][10]


Bedrock uses the zlib library to compress data, such as in the SGZip method.

Concurrency Control[11][12][04]


Bedrock inherits concurrency control from SQLite. SQLite maintains page locks using Two-Phase Locking.

However, Bedrock has its own proprietary synchronization engine to support concurrency over multiple servers.

Bedrock's synchronization engine is a private distributed general ledger, i.e, a private blockchain. Each thread has an internal table called journal, which has 3 columns called id, query, hash. Each time a query is committed to the database, a new row is inserted into the journal. The new row records the query, and calculates the new incremental hash based on the previous row. When a server connects to a cluster, the most recent id and hash are broadcasted. If two servers disagree on the the hash corresponding to the id, then they know that they have "forked" at some point and stop communicating with each other. A Paxos-based election scheme decides which fork stands up to the new master.

Since Bedrock supports multi-threaded writes, it is prone to write conflicts. This is addressed by "sharding" the table, and querying all the journal tables in a UNION whenever the database is to be viewed as one.

Data Model[01][13]


Bedrock inherits the data model from SQLite, which supports a relational data model.

Foreign Keys[02]


Same as SQLite

Bedrock inherits foreign key support from SQLite. To enable foreign key support, SQLITE_OMIT_FOREIGN_KEY and SQLITE_OMIT_TRIGGER must not be defined at runtime.

Hardware Acceleration[01]


Bedrock is written for modern hardware with SSD-backed RAID drives and RAM file caches. It was made to be "simple", not requiring any esoteric hardware hacks as done by other systems. Hence, there is no standard hardware acceleration option.

Indexes[02]


Same as SQLite

Bedrock inherits index support from SQLite. The default index is a B Tree. There is an option to use an R Tree for range queries, in which case SQLITE_ENABLE_RTREE needs to be defined.

Isolation Levels[11][14]


Same as SQLite

Bedrock inherits isolation level support from SQLite. The default behaviour is Serializeable. Snapshot Isolation can be implemented by setting PRAGMA journal_mode = WAL. If PRAGMA read_uncommitted = True along with the setting fro Snapshot Isolation, the isolation level becomes Read Uncommitted.

Joins[11][02][15]


Bedrock inherits join support from SQLite. SQLite uses nested loop joins, and has been criticised previously for its slow performance. SQLite supports Sort-Merge joins over unique keys.

Logging[11][02][16]


Bedrock inherits logging support from SQLite. SQLite uses page-level logging. It supports Write-Ahead Logging, which can be used by setting PRAGMA journal_mode = WAL.

Query Compilation[17]


Same as SQLite

Bedrock data is stored in SQLite databases. The compilation is hence the same as in SQLite. To execute a SQL statement, it is first compiled into a byte-code program using one of the sqlite3_prepare% methods. Once the code has been generated, it is run by a virtual machine.

Query Execution[18]


Same as SQLite

Bedrock uses the same query execution model as used by SQLite. The Squery method in Bedrock calls the sqlite3_exec method, which is a one step query execution interface.

Query Interface[19]


Bedrock queries can be any SQLite compatible query. The result is returned in an HTTP-like / JSON format, as per user request. It also support the MySQL protocol, and hence the user can continue using the MySQL client of their choice. It also provides a PHP binding that one can use to work with it from the shell itself.

Storage Architecture[20][02]


Same as SQLite

Bedrock has SQLite embedded in it, and reads / writes to ordinary disk files.

Storage Model[11]


Same as SQLite

Bedrock stores data in a SQLite database. This stores data in a row-wise, where each row is referred to as a tuple. The tuples are stored contiguously on each page, and can be stored across multiple pages.

Stored Procedures[21]


Bedrock uses C++ as its primary stored procedure language. Hence, it can be inferred that it supports stored procedures.

System Architecture[17]


Bedrock is built on top of SQLite and has the same system architecture.

Views[22][23][24]


Same as SQLite

Bedrock inherits view support from SQLite. SQLite supports virtual views. Whether SQLite supprots Materialized views is a topic of debate - while the official documentation claims it supports materialized views, the definition of materialized as used by SQLite seemed to vary.

Citations

27 sources
  1. Bedrock by Expensify bedrockdb.com
  2. GitHub - Expensify/Bedrock: Rock solid distributed database specializing in active/active automatic failover and WAN replication · GitHub github.com
  3. Bedrock/docs at main · Expensify/Bedrock · GitHub github.com
  4. Bedrock by Expensify bedrockdb.com
  5. Your Database is Your Prison — Here’s How Expensify Broke Free firstround.com
  6. Checkpoint a database sqlite.org
  7. Write-Ahead Logging sqlite.org
  8. zlib - Wikipedia wikipedia.org
  9. zlib Usage Example zlib.net
  10. Bedrock/libstuff at main · Expensify/Bedrock · GitHub github.com
  11. https://dbdb.io/db/sqlite dbdb.io
  12. Bedrock by Expensify bedrockdb.com
  13. SQLite Home Page sqlite.org
  14. Isolation In SQLite sqlite.org
  15. http://sqlite.1065341.n5.nabble.com/Join-performance-in-SQLite-td50946.html nabble.com Dead — Check Archive
  16. https://documentation.xojo.com/404.html xojo.com
  17. Architecture of SQLite sqlite.org
  18. One-Step Query Execution Interface sqlite.org
  19. Bedrock by Expensify bedrockdb.com
  20. https://www.loc.gov/preservation/digital/formats/fdd/fdd000461.shtml loc.gov
  21. Bedrock by Expensify bedrockdb.com
  22. https://www.whoishostingthis.com/compare/sqlite/optimize whoishostingthis.com Dead — Check Archive
  23. SQLite - Views tutorialspoint.com Dead — Check Archive
  24. SQLite Create View: Learn How to Create View in SQLite sqlitetutorial.net
  25. First commit github.com
  26. https://github.com/Expensify/Bedrock/commit/b3e3ad6b14fb7631c3af7e8b8d5e5d05589b9447 github.com
  27. https://github.com/Expensify/Bedrock/commit/df739c7cef23c79bcc41e2bfe8b30fcfe6abee36 github.com
Revision #43 Last Updated: