Table of contents
1.
Introduction
2.
What is RedisTimeSeries?
3.
Features of RedisTimeSeries
4.
Memory Model
5.
Run-time configuration
5.1.
Passing Configuration Options During Loading
5.2.
RedisTimeSeries Configuration Options
5.2.1.
₹COMPACTION_POLICY {policy}
5.3.
RETENTION_POLICY
5.4.
CHUNK_TYPE
5.5.
NUM_THREADS 
5.6.
DUPLICATE_POLICY
5.7.
Precedence Order
6.
Frequently Asked Questions
6.1.
Can Redis store integers?
6.2.
How is the data stored in Redis?
6.3.
Is Redis atomic?
6.4.
Does Redis save to disk?
6.5.
Can Redis lose the data?
7.
Conclusion
Last Updated: Mar 27, 2024
Medium

RedisTimeSeries

Author ANKIT MISHRA
0 upvote

Introduction

Redis (Remote Directory Server) is a data structure that runs in memory. It's a disk-permanent key-value database that can deal with a wide range of data structures and types. This means that, in addition to mapping key-value-based strings for storing and retrieving data (like traditional databases do), Redis also supports more advanced data structures like lists, sets, and so on. As we go along, we'll look at a data structure called RedisTimeSeries. Let's Start!

What is RedisTimeSeries?

RedisTimeSeries is a Redis module that provides Redis with a time series data structure.

A Redis time series consists of the following:

  • Raw Samples
    Each raw sample is consist of a {time tag, value} pair.
    Time tags have been measured in milliseconds since January 1st, 1970, at 00:00:00.
    The client can specify time tags or fill them automatically by the server.
    64-bit floating-point values.
    The intervals between the time tags can be a constant or variable.
    Raw samples can be reported in order or out of order.
    The duplication policy for samples with identical time tags can be set: block/first/last/min/max/sum.
  • Retention period
    Raw samples older than the retention period (relative to the raw sample with the highest time tag) will be discarded.
  • Series Metadata
    A set of name-value of pairs (e.g., room = 3; sensorType = 'xyz').

RedisTimeSeries supports the cross-time-series commands. One can, for example, aggregate the data overall sensors in the same room or all sensors of the same type,

Examples:

Sensor data: Like temperatures or the fan velocity for a server in a server farm.

Past Prices of a stock.

A number of vehicles passing through the given road on a Time (count per 1-minute timeframe).

Features of RedisTimeSeries

  • Inserts with a high volume and low latency.
  • Query by beginning and ending times.
  • For any time bucket, aggregated queries (Min, Max, Avg, Sum, Range, Count, First, Last, STD.P, STD.S, Var.P, Var.S, twa).
  • The maximum retention period can be configured.
  • Compactions- Aggregated time series that are automatically updated.
  • Each time series has labels (name-value pairs) that can be queried using the secondary index.

Memory Model

A time series is a collection of memory bits connected together. Each chunk has a predetermined sample size. Each sample is a 128-bit tuple, with the timestamp and value separated by 64 bits.

Run-time configuration

A few run-time configuration settings for RedisTimeSeries should be determined while loading the module. More options will be introduced over time.

Passing Configuration Options During Loading

In general, passing the configuration options is done by appending the arguments after the --loadmodule argument in the command line, loadmodule configuration directive in a Redis config file, or the MODULE LOAD command. For example:

In redis.conf:

loadmodule redistimeseries.so OPT1 OPT2

From redis-cli:

127.0.0.6379> MODULE load redistimeseries.so OPT1 OPT2

From command line:

$ redis-server --loadmodule ./redistimeseries.so OPT1 OPT2

RedisTimeSeries Configuration Options

₹COMPACTION_POLICY {policy}

Default compaction/downsampling rules for the newly created key with TS.ADD.

A semicolon (;) separates each rule, which is made up of many fields separated by a colon (:):

The Aggregation function - avg, sum, min, max, count, first, last

Time bucket duration - number and time representation (Example for 1 minute to be: 1M)

m = millisecond

M = minute

s = seconds

d = day

retention time = in milliseconds

Example:

max:2M:2h - Aggregate using max over 2 minutes and retain the last 2 hour

$ redis-server --loadmodule ./redistimeseries.so COMPACTION_POLICY max:1 m:2h;min:20s:5d:10d;last:5M:10ms;avg:2h:10d;avg:3d:100d

RETENTION_POLICY

Maximum age for the samples compared to last event time (in milliseconds) per key. This configuration will be used to set the default retention for newly created keys that do not have an override.

Default

0

Example

$ redis-server --loadmodule ./redistimeseries.so RETENTION_POLICY 10

CHUNK_TYPE

Default to chunk type for automatically created keys when COMPACTION_POLICY has been configured. Possible values are: COMPRESSED or UNCOMPRESSED.

Default

COMPRESSED

Example 

$ redis-server --loadmodule ./redistimeseries.so COMPACTION_POLICY max:1m:1 h; CHUNK_TYPE COMPRESSED

NUM_THREADS 

The maximal number of the per-shard threads for every cross-key query when using the cluster mode (TS.MRANGE, TS.MGET, and TS.QUERYINDEX). The values must be equal to or greater than 1. Note that the increase of this value may either increase or decrease the performance!

Default

3

Example

$ redis-server --loadmodule ./redistimeseries.so NUM_THREADS 4

DUPLICATE_POLICY

A policy that will handle duplicate samples. Must require the following possible policies:

BLOCK - A error will occur for any out of order sample.

FIRST - Ignore the newly provided value.

LAST - Override with the latest value.

MIN - Only override if the value is encountered lower than the existing value.

MAX - Only override if the value provided is higher than the already existing value.

SUM - If a previous sample already exists, add the newly provided samples to it to equal the updated value (previous + new). If no previous sample exists, then set the updated value to be equal to the new value.

Precedence Order

Since the duplication policy can be provided at the different levels, the actual precedence of the used policy will be:

TS.ADD input

Key level policy

Module configuration (AKA database-wide)

Default configuration

The default policy for database-wide is BLOCK. New and pre-existing keys will conform to the database-wide default policy.

Example

$ redis-server --loadmodule ./redistimeseries.so DUPLICATE_POLICY LAST

Frequently Asked Questions

Can Redis store integers?

Redis maintains integers in their integer representation, so there is no overhead for keeping the string representation of the integer in string values that hold an integer.

How is the data stored in Redis?

Redis is a non-relational key-value store that runs in memory (sometimes referred to as a data structure server). This implies that it stores data using keys and values – think of it as a massive dictionary that stores information using words and definitions.

Is Redis atomic?

Redis transactions are atomic as well. Atomic indicates that either all or none of the commands are processed.

Does Redis save to disk?

Redis maintains snapshots of the dataset on the disc by default in a binary file named dump. You can set up Redis to save the dataset every N seconds if there are at least M changes, or you can call the SAVE or BGSAVE commands manually.

Can Redis lose the data?

Redis is regarded as a fast cache rather than a database that ensures data consistency. As a result, its use cases are often different from those of actual databases. For example, you can store sessions, performance counters, or anything else in it with unrivalled performance and no actual loss in the event of a crash.

Conclusion

In this article, we have extensively discussed various the RedisTimeSeries data type in Redis, and their configuration and use cases under various cases of data. For more such blogs, you can visit our Blogs section, also read Tools for RedisStreamsNo SQL DataBase.

If you want to learn more, check out our articles on Code studio. Do upvote our blog to help other ninjas grow.

“Happy Coding!”

Live masterclass