Checkpoints

Consistent

MongoDB uses a combination of consistent checkpoints and a write ahead log known as the database journal. Every 60 seconds, a consistent version of the data is written to disk. Between these checkpoints, all changes are recorded in a write ahead log, allowing the database, (on recovery) to automatically roll forward from the last checkpoint to the last journal flush. Writes can wait for WAL (write ahead log) journal flush to ensure 100% durability.

Concurrency Control

Two-Phase Locking (Deadlock Prevention) Optimistic Concurrency Control (OCC)

Data in MongoDB is stored in the underlying WiredTiger storage engine. Whilst WiredTiger implements optimistic locking for concurrency control, the MongoDB databases operations above it use automatic retry to present pessimistic concurrency control for individual documents. Pessimistic locking also exists at the collection/table and database level for some administrative operations. MongoDB also includes traditional ACID transaction semantics for writes with begin, commit, and rollback commands. These can be used when an update needs to modify more than one record, although in a document model database, this is less common.

Data Model

Document / XML

MongoDB stores data in a binary representation called BSON (Binary JSON). The BSON encoding extends the popular JSON (JavaScript Object Notation) representation to include additional types, such as int, long, date, floating point, and decimal128. BSON documents contain one or more fields, and each field contains a value of a specific data type, including arrays, binary data, and sub-documents. Documents that tend to share a similar structure are organized as collections. With the MongoDB document model, data is more localized, which significantly reduces the need to JOIN separate tables. The result is dramatically higher performance and scalability across commodity hardware as a single read from the database can retrieve the entire document containing all related data.

Indexes

B+Tree

Fundamentally, indexes in MongoDB are similar to indexes in other database systems. MongoDB defines indexes at the collection level and supports indexes on any field or subfield of the documents in a MongoDB collection. MongoDB provides a number of different index types to support specific types of data and queries. The unique indexes will reject insertion or update with existing value. The compound indexes are useful for queries that specify multiple predicates. For fields that contain an array, each array value is stored as a separate index entry. Time to live indexes allow the user to specify a period of time, after which the data is automatically deleted from the database. Geospatial indexes optimize queries related to location data within a two-dimensional space. Partial indexes only index the documents in a collection that meet a specified filter expression. MongoDB uses b-tree for the data structure of the index. MongoDB also includes the ability to create a clustered index on the primary key of tables to organise the underlying record storage.

Isolation Levels

Read Uncommitted Read Committed Snapshot Isolation Repeatable Read

Reads from MongoDB are only of fully committed transactions by default, providing true ACID properties. However, the isolation level can be adjusted by setting the appropriate “read concern” level when performing read operations. The default is to show transactions fully committed that are persisted on the current primary node, with a read concern of "majority", meaning the transaction is replicated to a majority of high availability copies. “linearizable” read concern in MongoDB ensures that read operations reflect the latest committed state of the database, providing a linearizable view of the data.

Joins

Nested Loop Join

MongoDB supports left outer joins via the keyword $lookup. $lookup performs a nested loop join with internal invariant caching. In document databases such as MongoDB, it is common practice to use a JOIN only for fairly rapidly changing dimensions and the additional document modeling features to reduce the number of joins required. For example, a repeating subgroup of attributes can be represented without need to use an additional table. Joins can also be performed using a combination of $unionWith and $group.

Logging

Physiological Logging

MongoDB maintains a fixed sized, on-disk circular buffer - called the oplog - of all logical database operations. This is used for replication and can also be observed to provide an event stream of database operation using the watch() command. The oplog is the only data structure that has write ahead logging enabled so in the event of failure, a Write Ahead Log (Journal) rolls the oplog forward from the latest full consistent disk snapshot. Then, the oplog operations are replayed on the data to bring the whole database up to sync with the point of failure. At this point, if the instance is part of a highly available replica set, then additional transactions are recovered from the current primary node and reconciled by rolling back to the last point of common truth and playing transactions to catch up. All this is fully automatic.

Query Compilation

JIT Compilation

Queries in MongoDB are not parsed from text like an SQL query but are supplied from the client in the form of a binary object representation of the required operations and parameters. At its simplest, this could be described as query-by-example, although the object model allows for arbitrary complex comparison functions. These structured query objects are traversed at run-time to build a processing tree for the underlying query engine.

Query Execution

Tuple-at-a-Time Model

Within a single instance, MongoDB performs linear execution of simple retrievals but can perform parallel chained operations when using a pipeline query. When querying a collection (table) that has been partitioned across multiple hosts, each partition can be queried in parallel.

Query Interface

Custom API

MongoDB provides native drivers for all popular programming languages and frameworks to make development natural. Supported drivers include Java, Javascript, C, C++, .NET, Python, PHP, Ruby, Scala, and others, in addition to community-developed drivers. MongoDB drivers are designed to be idiomatic for the given language.

The mongo shell (mongosh) is an interactive JavaScript REPL that allows users to interact with the database by immediately executing individual lines of code in a user interface similar to a traditional SQL shell. MongoDB Compass is a GUI for MongoDB. Offering schema exploration and management, Compass allows DBAs to modify documents, create validation rules, and optimize query performance by visualizing explain plans and index usage.

Storage Architecture

Hybrid

MongoDB by default is a disk-oriented database, with all reads and writes going via a database cache. There is no practical limit to database size, with only the most recently used elements being held in RAM in the cache and older data being auto-evicted as required. The database cache is configurable but typically approximately half the RAM on the server, with the rest being used for query process and OS tasks, such as the underlying OS page cache. MongoDB has the option of running as an in-memory database, in which case, the maximum storage capacity is equal to the configured database cache size - normally 50% of RAM - and can be adjusted to be larger.

Storage Format

Custom

MongoDB stores data by default on disk using the BTree/B+Tree hybrid WiredTiger storage format and engine. WiredTiger represents each BTree as a single file, with the file containing both a consistent data snapshot referenced from the root node and any blocks written to disk but not yet included in a consistent snapshot. Collections in MongoDB are organized as BTrees (technically a variant B+Tree where data lives in leaf nodes but there are no links explicitly between leaf nodes) where the key is an auto-assigned incremental key called an "Identity" and the value is a BSON document. Indexes are also WiredTiger trees, with the key being the index key and the value being the identity of the record it points to. Individual blocks in WiredTiger data files are compressed by default and can optionally be encrypted for protection at rest.

Storage Model

Custom

MongoDB stores BSON serialized objects (documents) in the database, making it a row-based system. It also has a Time Series option, where specific collections of individual row-like documents are converted to and stored in specially formatted hidden documents in a clustered, columnar format, providing greater compression and read performance for time series values.

Stored Procedures

Not Supported

MongoDB does not support stored procedures in the traditional sense. MongoDB's aggregation pipelines are a turing-complete language for manipulating data, which can be used to perform analysis, generate sample records, or perform complex updates or processing on the server without the need to retrieve the data and perform these operations client-side. Aggregation pipelines can also execute JavaScript code on the server using $function if required, although this is not recommended when compared to native aggregation code. Although aggregation pipelines can be stored as records in MongoDB, to execute them, a client needs to retrieve the pipeline code and then send it back to the server as an aggregation request. They cannot be executed explicitly the way a stored procedure can. MongoDB has, in the past, suggested that business logic should sit in a layer above the database server and as a company, offers cloud-hosted triggers and functions as a service to do this.

System Architecture

Shared-Nothing

MongoDB systems consist of replica sets: groups of identical instances hosted on different underlying servers, serving the same data and replicating continuously to ensure high availability as well as providing additional query capacity. Larger data sets can be range or hash partitioned over multiple replica sets in a process known as sharding. In a sharded cluster, the values of one or more fields known as the shard key determine which replica set hosts that specific record.

Views

Virtual Views

MongoDB supports read-only, non-materialized views, defined as an aggregation and a collection or view on which it will be run at query time. When a view is the source collection from another view, it allows us to execute a chained aggregation. Views are read-only. Write operations on views will error.