Table of contents
1.
Introduction
2.
Redis Time Series
2.1.
Time Series With Redis Data Structures 
2.1.1.
Streams
2.1.2.
Sorted Sets
2.2.
Capabilities of Redis Time series
2.3.
Benefits of Redis Time Series
2.4.
Features of Redis Time Series
3.
Out of Order Performance Considerations in Reference 
3.1.
Analysis of Compressed Chunks
3.2.
Analysis of uncompressed Chunks
4.
Frequently Asked Questions
4.1.
Name all the main four components of the Time Series.
4.2.
What is the maximum number of records redis can handle?
4.3.
What do you mean by white noise in time series?
4.4.
What do you understand by a random walk in time series?
5.
Conclusion
Last Updated: Mar 27, 2024

References of Time - series

Author Naman Kukreja
0 upvote
Career growth poll
Do you think IIT Guwahati certified course can help you in your career?

Introduction

For a company storing the data and analyzing it at an appropriate timestamp is necessary to compete. They need to understand the different needs of people with changing times, but if they don't, they will soon be out of competition and lose a significant amount of money and time.

To store and analyze the data at different timestamps, we can use time series and redis data structure, but we don’t know how to model redis core data structure with time series. Don't worry. You will learn everything about merging redis data structures with time series and references of time series while moving further with the blog, so without wasting any time further, let's get on with our topic.

Redis Time Series

In time series, the data is stored in a connected manner like LinkedList, with different nodes representing the different timestamps.

Time Series With Redis Data Structures 

There are mainly two ways to use time series in redis while using its data structures: streams and sorted sets. We will discuss both of them in this blog section:

Streams

Redis Streams are the most recently added data structure, and because of this, they are not frequently used for time series by the users. It consumes comparatively less memory than another method, i.e., sorted sets. It is implemented using a different method of radix trees known as Rax. Generally, Redis streams enhance the performance of reads and interactions compared to sorted sets. It is designed as a generic data structure, so it misses a toolset specific to the time series.

Downsides of this approach:

  • It only has one downside, i.e., it misses a time series toolset.

Sorted Sets

In sorted sets, the values are stored by their scores. In this, the score refers to the timestamp at which data is stored. We store the value of the timestamp followed by a colon and then the actual measurement, e.g.,, “ <timestamp>:<measurement>”. As it is set, every value stored is unique. They store the data in the form of a hash table, i.e., key-value pair.

Downsides of this approach:

  • These are not memory efficient.
  • It’s missing a time series toolset.
  • It takes more time to insert.

Capabilities of Redis Time series

The data is stored in memory chunks and uses radix tree implementation as a redis stream. Streams limit the number of messages by count.

  • Integrations: With the redis time series, you can integrate into several existing time series tools. We can have all the data inside monitoring metrics. With its remote data adapter, we can operate it from a remote distance.
  • Aggregation: While reading the data, you can aggregate it according to your requirement. During aggregation, it makes sure the data is traversal with low latency.
  • Secondary Indexing: You can only retrieve a time series using Redis' fundamental data structures if you know the exact key that holds the time series. Unfortunately, your application won't know the precise key it's looking for in many time series use cases. To extract the knowledge you need, these use cases often desire to query a number of time series related to each other in a few dimensions. To help with this, you could design your secondary index using core Redis data structures, but it would be expensive to develop and require you to manage edge cases to ensure the index is right.
  • Downsampling: Your data set will grow linearly over time if you want to keep all of your raw data points indefinitely. Downsampling can be used if your use case enables you to have less fine-grained data further back in time. Using a given aggregation function, you can keep fewer historical data points by aggregating raw data for a specific time range. RedisTimeSeries support the following aggregations: average, sum, min, max, capacity, count, first, and last.

Benefits of Redis Time Series

There are many benefits of using the redis tie series, and we will discuss some of them in this blog section:

  • Easy and Efficient: The most efficient and easiest way to store time series is redis. Downsampling, retention rules, and multi-key queries can be possible using some simple commands.
  • Tight integrations with popular tools: It rapidly integrates with the tools like Prometheus, Grafana, Telegraf, and StatsD for visualization and data integration in monitoring.
  • Tight Coupling with other Modules: Redis Time Series can work well with Redis Gears and RedisAI, enabling advanced use cases such as predictive maintenance and anomaly detection.

Features of Redis Time Series

The redis time series has many features, discussing some of them in this blog section.

  • Low latency reads High Volume inserts.
  • Secondary indexing for time-series indexing.
  • Configurable maximum retention period.
  • Aggregate queries for any time bucket.
  • Query by end-time and start-time.

Out of Order Performance Considerations in Reference 

We have to retrieve the data and the position from the main memory, so when an older timestamp of memory is inserted, then it will get inserted into the main memory, and when it is in the compressed form, then first we have to decode it before working on it. Ingest performance is very critical. 

We should be able to assess the impact of the backfilled ratio on our overall high-performance TSDB.

With the help of a benchmark, we can control key factors that manage the overall performance like the compression of the series, the out-of-order ratio, command pipelining, and the number of concurrent clients used. All the benchmarks run on AWS(Amazon Web Services), and they are tested by benchmarking testing infrastructure.

Analysis of Compressed Chunks

Given that a single out-of-order datapoint implies the whole decompression from the double delta of the entire chunk with compressed chunks, you should expect larger out-of-order writing overheads.

To increase the compressed performance, we reduce the size of the memory chunks as small as possible. Smaller chunks mean less double-delta decompression calculation and thus less overall impact, at the cost of a lower compression ratio.

  • If the database receives 1% of out-of-order samples, the overall ingestion effect will be 10%.
  • For large out-of-order sample values like 10% and 25%, the overall ingestion effect will be 35% to 75%.
  • The maximum drop available till now is 95% at any out-of-order ingestion.

Analysis of uncompressed Chunks

In the case of uncompressed chunks, the chunk size will not affect the overall out-of-order impact on ingestion. For example, the impact of ingestion will be identical for the memory chunk of 4096 bytes and 256 bytes.

  • For 1% of out-of-order samples, the ingestion rate will be very low or even unmeasurable.
  • For large out-of-order sample values like 10% and 25%, the overall ingestion effect will be 5% to 19%.
  • The maximum drop available till now is 45% at any out-of-order ingestion.

Frequently Asked Questions

Name all the main four components of the Time Series.

The four main components of time series are cyclic fluctuations, Seasonal Variations, Irregular Variations, and Secular trends.

What is the maximum number of records redis can handle?

Yes, Redis can handle up to 2^32 keys and 250 million keys at an instance.

What do you mean by white noise in time series?

White noise refers to the independent variables which are distributed identically with a mean of zero.

What do you understand by a random walk in time series?

It is another time series where the current observation equals the previous observation with a random step down or up.

Conclusion

In this article, we have extensively discussed the Redis Time Series references for both compressed and uncompressed chunks, along with how to integrate other time series with redis data structure, different capabilities, features, and benefits of redis time series.

We hope this blog has helped you enhance your knowledge of the reference of the Redis Time series. If you are interested in learning more about the redis time series and its development, then you must refer to this blog. Here you will get the complete idea about development in the redis time series with step by step explanation.

Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, React, JavaScript, System Design, etc. Enroll in our courses, refer to the mock test and problems; look at the interview experiences and interview bundle for placement preparations. Do upvote our blog to help other ninjas grow.

 “Happy Coding!”

Live masterclass