Compression[05][06][07][08]

The available encoders (compressors) are ZSTD, ZLIB, SNAPPY and in PLAIN. On a record-level base configuration it is done by updating Cell-Value with the desired encoder. On a page-level base that is applied at Cell-Store-Block it is done by defining the default configuration for a schema or by column-schema definition with the encoder to use. Transactions of Communications are configurable at a program specific (application level) with an application designated configuration property and file. Variable-Length integers are used for most of numerical primitives either long-term/persistent storage or intermediate run-time consumption.

Concurrency Control[05][09]

The option to define the maximum cell-versions to keep/retain in SWC-DB is done by configuring the column's Schema property "cell versions", whereas the order and the revision of a cell is determined by the Cell's definitions of timestamp and it's order DESC/ASC. The timestamp defaults to AUTO that is swcdbRanger application system's time and order to ASC. Additionally to the user-supplied timestamp SWC-DB cell has an internal cell-revision definition that let to determine the latest data that should be retain or removed.

The precision of timestamp used in SWC-DB is in nanoseconds. Whereas the configurable 'cell ttl' property of a column schema is in milliseconds. The Cell's TTL of the schema is applied against timestamp of a cell and at reach of TTL the cell is auto-considered as non-existing, depends on momentum and max-versions, by changing the Schema's cell-ttl before Compaction has under-gone It is a possibility the Cell to exist.

Data Model

Joins

Query Interface

Storage Architecture[10][11]

The File-System of SWC-DB is configurable and on current date supports Hadoop-JVM, Hadoop(Native C++), Ceph, Local, Custom (a user developed library) and FsBroker. Depends on SWC-DB release/build-type, filesystem/s can be an implementation or to be a dynamically loaded library. The FsBroker-Filesystem for SWC-DB applications is a filesystem-type whereas swcdbFsBroker(application) has an underlying fs-type configuration. The swcdbFsBroker can be a running on remote instances to the hosts running the swcdbManager/s or/and swcdbRanger/s. The Custom-Filesystem is a type name-holder for a user developed File-System library based on SWC::FS::FileSystem interface class (limitless of FS Impl. possibilities).

The SWC-DB base FS path is determined and defined by the configuration-properties --swc.fs.TYPE.path.root and --swc.fs.path.data and it is evaluated as swc.fs.TYPE.path.root="/var/opt/swcdb/" joined with swc.fs.path.data="your/cluster/name". As example to have FS-base at root folder of HDFS it is swc.fs.hadoop_jvm.path.root=/swcdb/ plus joined with swc.fs.path.data="or/no/sub/folders".

The data-topology(files/folders structure) within the SWC-DB base-path is based by column-id and range-id, that on path consist CellStores and CommitLog files while at any point one server is responsible for a range-id on column-id and of a path root. The CellStores are files storing Cells in serialized form that are after latest compaction whereas Commit-Log is many Fragments of current added data, the fragments are written on a (schema configurable log-rollout-ratio) threshold reach or on shutdown. SWC-DB uses structure ID folder-names and a single-folder won't have more than 2000 entries a folder.

System Architecture