Prior to Atlas, Netflix used an in-house tool called Epic and a commercial system. This was done since there was a huge surge in the number of metrics Netflix was monitoring between 2011 and 2014. The number of metrics they monitored increased from 2 million to 1.2 billion. They decided on building a system from scratch to suit their requirements.

Like Lucene, Atlas treats tags as documents and an inverted index is created for these tags to find the set of matching time series. The key building block for the inverted index is integer set implementation for which a roaring bitmap is used. Once a set of matches is found, the lookup for time-series data is effectively into a huge id - to - time series map.

Atlas does not support the full Relational Database style joins. But it does support simple joins in terms of binary math operations. Simple aggregates (like sum) generate a single value for a time series. If the time-series expression is grouped then there can be two cases, 1) If both the sides are grouped (they must be grouped on the same keys), then the corresponding entry on each side is found and the operation is applied to them 2) If only one of the time-series is grouped, then single result from the other side is applied to each result from the other side.

https://github.com/Netflix/atlas

https://github.com/Netflix/atlas

https://github.com/Netflix/atlas/wiki

Netflix

2014

Open Source