Storing and Processing Time Series Data - Time Series Databases - page 34

In normal operations, at the end of a time window, new in-memory structures are created,
and the now static old in-memory structures are used to create compressed data blobs to
write to the database. Once the data blobs have been written, the log files are discarded.
Compare the point in the data flow at which writes occur in the two scenarios. In the hybrid
approach shown in Figure 3-5 , the entire incoming data stream is written point-by-point to
the storage tier, then read again by the blob maker. Reads are approximately equal to writes.
Once data is compressed to blobs, it is again written to the database. In contrast, in the main
data flow of the direct blob insertion approach shown in Figure 3-6 , the full data stream is
only written to the memory cache, which is fast, rather than to the database. Data is not writ-
ten to the storage tier until it's compressed into blobs, so writing can be much faster. The
number of database operations is decreased by the average number of data points in each of
the compressed data blobs. This decrease can easily be a factor in the thousands.