Bedrock

View Current Viewing Revision #18 from 11/24/2019 12:42 p.m.

Bedrock is a distributed relational DBMS designed for geo-replication. It uses SQLite as its backing data storage and supports the MySQL wire protocol.

History

BedRock is the system that backs Expensify, the expense management company. It had been used for 8 years prior to being launched. It was originally created as an in house solution to the strict database constraints of financial institutions - response time within milliseconds, transaction logging and authentication, and replication of multiple servers.

Storage Model

N-ary Storage Model (Row/Record)

Concurrency Control

Multi-version Concurrency Control (MVCC)

Bedrock's synchronization engine is a private distributed general ledger, i.e, a private blockchain. Each thread has an internal table called journal, which has 3 columns called id, query, hash. Each time a query is committed to the database, a new row is inserted into the journal. The new row records the query, and calculates the new incremental hash based on the previous row. When a server connects to a cluster, the most recent id and hash are broadcasted. If two servers disagree on the the hash corresponding to the id, then they know that they have "forked" at some point and stop communicating with each other. A Paxos-based election scheme decides which fork stands up to the new master.

Since Bedrock supports multi-threaded writes, it is prone to write conflicts. This is addressed by "sharding" the table, and querying all the journal tables in a UNION whenever the database is to be viewed as one.

Data Model

Relational

Bedrock is a networking and distributed transaction layer built on top of SQLite. SQLite supports a relational data model.

Hardware Acceleration

Custom

Bedrock is written for modern hardware with SSD-backed RAID drives and RAM file caches. It was made to be "simple", not requiring any esoteric hardware hacks as done by other systems. Hence, there is no standard hardware acceleration option.