Memory Usage of Redis JSON

Introduction

Redis is used by millions of programmers. It is an open-source database, in-memory data store used as a database, streaming engine, message broker, and cache. It is a NoSQL database. To know more about NoSQL databases, visit this page NoSQL Databases. In this article, we will debug the memory consumption of Redis JSON. 

Redis Json Memory Usage

Redis is an in-memory key-value data store. Every key in Redis takes up some memory. Whenever we create a key, some memory is required to store the key name. Also, for each key, there is some per-key overhead that Redis uses. This is the minimum requirement, but then to store the value of that key, we require some more memory. Now, we will discuss the way it is stored in memory.

After deserializing JSON values, RedisJSON saves them as binary data. The deserialized values have more size than the serialized form. RedisJSON data type uses a minimum of 24 bytes(for 64-bit architecture) for each value stored. This can be seen and verified by running the following command on an empty string using JSON.DEBUG MEMORY command

127.0.0.1:6379> JSON.SET emptystring . '""'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY emptystring
(integer) 24

This minimum memory of 24 bytes is the same for all scalar values, but a string has a variable memory requirement. A string is a data type that has a variable length, so depending upon the length, the memory requirement also varies. Suppose you have the string "REDIS" the length of this string is 5, so we will need an additional memory of 5 bytes, but if the string were "JSON", then an additional memory of 4 bytes would be needed.

This can be seen below.

127.0.0.1:6379> JSON.SET foo . '"REDIS"'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY foo
(integer) 29
127.0.0.1:6379> JSON.SET demo . '"JSON"'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY demo
(integer) 28

Now, let's move on to containers; each empty container in Redis takes up to 32 bytes of minimum memory. This concept is demonstrated by running the following commands:

127.0.0.1:6379> JSON.SET obj . '{}'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY obj
(integer) 32
127.0.0.1:6379> JSON.SET arr . '[]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 32

The memory of 32 bytes is minimum for containers, but when dealing with containers containing some elements, the total memory consumption of the containers increases by the amount required to store each element. So, the actual size becomes more than 32 bytes for a container containing elements. 

Now when the memory assigned to a container gets taken up, it is reallocated to provide memory to store more elements. To avoid expensive reallocations, the memory of containers is grown by a scale of 2 up to a certain amount, and then each time, we grow the memory by a fixed amount.

A container that has a single scalar will take up the minimum amount of memory for the container set up, which is 32 bytes, and an additional 24 bytes for the scalar element. This adds up to taking 56 bytes of memory overall. The below command can be used to verify this point:

127.0.0.1:6379> JSON.SET arr . '[""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 56

A container for 2 elements will take 40 bytes of memory; 32 bytes for the container set up and 8 bytes for each pointer to an entry in the container. After this, a memory of 24*2 is used to store the two scalar elements. So, the total becomes 88 bytes.

127.0.0.1:6379> JSON.SET arr . '["", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 88

Up to now, the picture is clear. To store 3 items, we need to assign enough memory to store 4 elements as the container space grows by a scale of 2. So, the container will take 32 + 8*3, and each pointer and element will take 24*3 bytes of memory. 

127.0.0.1:6379> JSON.SET arr . '["", "", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 128

Adding another scalar will not increase the container memory as we have already scaled for 4 items. Adding a fifth element will again scale the container to store 8 elements, and the total memory consumed by the container will grow.

127.0.0.1:6379> JSON.SET arr . '["", "", "", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 152
127.0.0.1:6379> JSON.SET arr . '["", "", "", "", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 208

The table below shows the size (in bytes) of a few test files on disk and when stored using RedisJSON. 

Note: In the latest version of Redis, when we delete values from containers, it does not free the container's allocated memory. 
You can also read about the memory hierarchy.