Sqrrl
Sqrrl is a graph database management system (DBMS) . It uses secure, adaptable NoSQL built from Apache Accumulo. Originally founded by former employees of the National Security Agency (NSA), Sqrrl focuses on cybersecurity.
The main product from Sqrrl is Sqrrl Enterprise, which is a solution for enterprise users on top of Accumulo that can handle massive structured and/or unstructured data sets. Sqrrl is aimed towards users who want to analyze data for security purposes. A Sqrrl whitepaper describes Sqrrl Enterprise as a threat hunting tool that uses large amounts of data with linked data analysis to aid users going through the 'hunting loop'. In particular, Sqrrl manages data and can display it to users raw or in the form of visualizations for analytics, finding threat patterns, or for further investigation.
Sqrrl has native Hadoop integration and supports multi-structured data.
- https://www.crunchbase.com/organization/sqrrl#section-overview
- https://db-engines.com/en/system/Neo4j%3BSphinx%3BSqrrl
- https://sqrrl.com/blog/
- https://sqrrl.com/media/Framework-for-Threat-Hunting-Whitepaper.pdf
- https://database.guide/list-of-graph-database-management-systems/
- https://datafloq.com/sqrrl/
History
Sqrrl development began in 2011. As a company, it was founded by Ely Kahn and Oren J. Falkowitz who worked in cybersecurity before creating Sqrrl. Sqrrl was built on Apache Accumulo. Customers using Sqrrl came from both the private and public sector.
In January of 2018, Sqrrl was acquired by Amazon Web Services (AWS). Existing users of Sqrrl were able to continue using the software without any changes. In late November 2018, Sqrrl co-founder Ely Kahn (who is now a Security Strategist at AWS) commented on the release of AWS Security Hub, which includes visualizations and security remediation recommendations. However, there were no statements made explicitly about Sqrrl's integration with AWS.
- https://www.crunchbase.com/organization/sqrrl#section-overview
- https://twitter.com/ElyKahn
- http://archive.boston.com/business/technology/innoeco/2012/08/new_database_startup_sqrrl_rai.html
- https://gigaom.com/2015/02/18/nsa-linked-sqrrl-eyes-cyber-security-and-lands-7m-in-funding/
- https://www.youtube.com/watch?v=K3PPzf9ePko&feature=youtu.be&ab_channel=AmazonWebServices
Data Model
After data is ingested, the Sqrrl Enterprise product details each piece at a cell level. Data can be a key/value pair or a field in a JSON document. Sqrrl then uses secondary indexing techniques to store the data in Apache Accumulo in its native form. This flat architecture allows Sqrrl to handle data of different structures.
In 2016, a presentation by Sqrrl stated that the version of Accumulo used by Sqrrl Enterprise stores sorted key-value pairs. Sqrrl then combines the data into graph storage form, using a linked data model.
Sqrrl is therefore considered a graph DBMS because it both represents data to users in a graphical format and is schema-free. Custom iterators for Accumulo developed by Sqrrl allow users to complete graph analysis on the data.
- https://hortonworks.com/blog/rebalancing-security-equation/
- https://db-engines.com/en/system/Sqrrl
- https://www.slideshare.net/HadoopSummit/sqrrl-threat-hunting-platform
- https://arstechnica.com/information-technology/2013/06/secret-sqrrl-nsa-spin-off-company-releases-data-mining-tool/
- https://venturefizz.com/blog/sqrrl-in-cybersecurity-knowledge-is-power
Concurrency Control
Multi-version Concurrency Control (MVCC)
Concurrency control is implemented in Sqrrl in a way that is not publicly known. Sqrrl supports concurrency control by doing atomic updates on data entities.
System Architecture
With native Hadoop integration, Sqrrl Enterprise ingests data. It is encrypted and labeled, then indexed and stored in Sqrrl's version of Apache Accumulo within the Hortonworks Data Platform (HDP). Sqrrl and HDP, together, store multi-structured data in its native format. This allows users of Sqrrl to make queries of many different kinds. Sqrrl is considered a distributed database.
Compression
Delta Encoding Bit Packing / Mostly Encoding
Based on the implementation of Accumulo, Sqrrl likely uses several types of compression. GZip and LZO are two compression algorithms used by Accumulo. In addition, Accumulo uses delta encoding by resending information if it is the same as prior sent information. Delta encoding is also used when storing sequential information in the table rows.
Website
Developer
Sqrrl Data, Inc.
Country of Origin
Start Year
2012
Acquired By
Amazon
- https://www.cnbc.com/2018/01/23/amazons-cloud-business-acquires-sqrrl-a-security-start-up-with-nsa-roots.html
- https://sqrrl.com/
Project Type
Written in
Supported languages
ActionScript, C, C#, C++, Cocoa, Delphi, Erlang, Go, Haskell, Java, JavaScript, Ocaml, Perl, PHP, Python, Ruby, Smalltalk