Configuring and managing Redis in production
The following subheadings will guide you through the configuration of Redis in production.
Redis setup hints
- Redis is deployed using the Linux operating system. It is also tested heavily on OS X and tested from time to time on OpenBSD systems and FreeBSD.
- Next is to change the overcommit memory setting to '1' of the Linux Kernel.
-
you need to add vm.overcommit_memory = 1 to /etc/sysctl.conf.
-
Then run the command sysctl vm.overcommit_memory=1 for this to take committed effect immediately.
- Also, make sure that Redis won't be affected by the Linux kernel feature, huge transparent pages. Otherwise, it will impact both memory usage and latency significantly negatively.
-
This step is accomplished with the following command: echo madvise > /sys/kernel/mm/transparent_hugepage/enabled.
- Make sure to set up a swap in your system. If Linux does not have a swap and your Redis instance accidentally consumes too much memory, either the Redis will crash when it is out of memory, or the Linux kernel OOM killer will kill the Redis process.
- Set an explicit "maxmemory" options limit in your instance to ensure that it will give us a report of the errors instead of failing when the system memory limit is near to being reached. Now the main point is that "maxmemory" should be set by calculating the overhead for Redis, other than data and the fragmentation overhead. Let's say you have 10 GB of free memory; put it to 8 or 9. If you are using Redis in a very write-heavy application(using a lot of resources) while rewriting the AOF log or saving an RDB file on disk, there is a high chance that Redis may use up to 2 times the memory normally used.
- The additional memory used by the Redis is proportional to the number of memory pages modified by writes during the saving process.
- Use daemonize no when running under daemontools.
- Make sure to set up some non-trivial replication backlog, which must be proportionate to the amount of memory Redis is using.
- If you use replication, Redis will need to perform RDB(which consists of all user data stored in an internal, compressed serialization format) saves even if you have persistence disabled (this doesn't apply to diskless replication).
- If you are using replication(non-blocking on the master side), they will try to maintain an exact copy of the master, so replicas will be wiped if a master restarts with an empty data set. So make sure that either your master has persistence enabled or that it does not automatically restart on crashes. Replicas
- Redis does not require authentication and listens to all the network interfaces by default, which results in a big security issue if you leave Redis exposed on the internet or other places where attackers can reach it.
- To assist in troubleshooting, see the "LATENCY DOCTOR" and "MEMORY DOCTOR" command.
Running Redis on EC2
- Use hardware virtual machine(HVM) based instances, not paravirtual(PV) based instances.
- Avoid old instances families; for example: use m3.medium with HVM.
- Sometimes, EBS volumes have high latency characteristics. Therefore Redis's persistence with EC2 EBS volumes needs to be handled with care.
- You can try the new diskless replication if you have issues when replicas synchronize with the master.
Upgrading/ Restarting a Redis instance without downtime
Redis is designed to be a very long-running process in your server.
Many configuration options can be modified without a restart by simply using the command -- "CONFIG SET."
You can also switch from AOF(append-only file feature) to RDB(Redis Database Backup file) snapshots persistence, or the other way around, without restarting Redis.
However, a restart is mandatory from time to time. For example, when you need to modify some configuration parameter that is currently not supported by the "CONFIG" command or upgrade the Redis process to a newer version.
The following steps provide a way that is commonly used to avoid any downtime.
- Setup your new Redis instance as a replica of your current Redis instance. To do so, you need a server that has enough RAM to keep two Redis instances running simultaneously.
- What can be the reason for the replica not being able to start? If you use a single server, ensure that the replica is started in a different port than the controller instance.
- Wait for the initial replication synchronization to complete (check the replica's log file).
- Using "INFO," you need to ensure that the master and replica have the same number of keys. Ensure that the replica is working as per your wish and is replying to your commands entered(Use "Redis-cli" )
- Using "CONFIG SET slave-read-only no." You can write to the replica.
- Configure all your clients to use the replica. And you need to ensure that no client can write to the old master during the switch using the "CLIENT PAUSE" command.
- Once you are sure that the master is no longer receiving any query (Check this with the MONITOR command), Use the "REPLICAOF NO ONE" command to elect the replica to master using the, and then shut down your master.
Suppose you want to use Redis Sentinel or Redis Cluster, which provides high availability for Redis. In that case, the simplest way to upgrade to newer versions is to upgrade one replica after the other. It will be easy for you to perform a manual failover to promote one of the upgraded replicas to master and promote the last replica.
Frequently Asked Questions
What is Redis?
The open-source, in-memory data store is used by millions of developers as a database, cache, streaming engine, and message broker.
What are AOF and RDB in Redis?
You can enable the Redis append-only file feature (AOF) if you require data durability. While Redis Database Backup file(RDB) file is a dump of all user data stored in an internal, compressed serialization format.
Differentiate between Redis Cluster and Redis Sentinel?
Redis Sentinel provides high availability for Redis. You can create a Redis deployment that resists certain kinds of failures without human intervention.
Redis Cluster provides a way to run a Redis installation(no central architecture) where data is automatically split across multiple Redis nodes.
Conclusion
Redis is an open-source, in-memory data structure that provides data structures such as hashes, lists, sets, strings with range queries, bitmaps, hyperloglogs, sorted sets, geospatial indexes, and streams. It is used as a database, message broker, cache, and streaming engine.
Through this article, we got an idea about the administration of Redis, i.e., how to configure and manage Redis in production.
Click here to read about MongoDB, DataBases & list of 100 SQL problems.
If you wonder how to prepare data structures and algorithms to do well in your programming interviews, here is your ultimate guide for practicing and testing your problem-solving skills on Coding Ninjas Studio.
Happy Learning!!!