Table of contents
1.
Introduction
2.
Scaling
2.1.
Horizontal Scaling
2.2.
Vertical Scaling
3.
Horizontal Scaling with Redis Cluster
3.1.
Adding New Nodes to an Existing Cluster
3.2.
Adding a New Master Node
3.3.
Resharding the Cluster
3.4.
Adding a New Replica Node
3.5.
Removing Nodes from Existing Cluster
4.
FAQs
4.1.
What is Scaling in DBMS?
4.2.
What is Vertical Scaling?
4.3.
Which is Horizontal Scaling?
5.
Conclusion
Last Updated: Mar 27, 2024

Scaling

Author Rajat Agrawal
0 upvote
Career growth poll
Do you think IIT Guwahati certified course can help you in your career?

Introduction

In a database management system, scaling refers to the ability to increase a database system's capacity to accommodate larger queries and/or store more data without sacrificing performance. We either compress or enlarge the system to match the predicted needs during the scaling process. The scaling operation can be accomplished by providing resources to the current system to match the lower expectation, or by adding a new system to the existing one, or by doing both.

Let’s learn about scaling and its type in-depth.

Scaling

Scaling refers to the ability to increase a database system's capacity to accommodate larger queries and/or store more data without sacrificing performance.

There are two types of scaling:-

1.) Horizontal Scaling

2.) Vertical Scaling

Horizontal Scaling

Horizontal scaling is when new server racks are added to an existing system to match increasing expectations.

Consider the existing system, which consists of a rack of servers and resources. When the old system fails to fulfill anticipated needs, and those needs cannot be met by just expanding resources, we must deploy entirely new servers. This is considered as horizontal scaling. Horizontal scaling is the concept of adding more machines to our resource pool.

Vertical Scaling

Vertical scaling is the process of adding new resources to an existing system in order to fulfil a goal.

Consider the existing system, which consists of a rack of servers and resources. Vertical scaling is used when an existing system fails to fulfil predicted demands, and the expected needs can be met by simply increasing resources.

Vertical scaling is based on the concept of enhancing existing systems by adding more processing power (CPU, RAM).

Vertical scaling is both easier and less expensive than horizontal scaling. It also takes less time to repair.

Horizontal Scaling with Redis Cluster

Redis scales horizontally with a Redis Cluster deployment topology.

It provides a way to automatically run a Redis installation with data sharded across multiple Redis nodes. It also provides some level of availability during partitions or the ability to keep running operations even if some nodes fail or cannot communicate. In the event of a larger failure, the cluster will shut down.

Let's learn how to do horizontal scaling with the Redis Cluster step by step.

Adding New Nodes to an Existing Cluster

A new node can be added as a master node or a slave node when adding it to an existing cluster (In Redis, slave nodes are generally called replicas as they hold the replicated slots that their masters have).

Adding a New Master Node

The first thing you should do is create an empty node.

It's as simple as starting a new node on port 6007 with the same configuration as the other nodes, except for the port number. Thus, to keep with the previous nodes' setup, we create a redis.conf file similar to the other nodes but with 6007 as the port number, and then start the server with the following command:

../redis-server ./redis.conf

The server should be up and operating at this point.

We can now use redis-cli to add the node to the existing cluster as usual.

./redis-cli --cluster add-node 127.0.0.1:6007 127.0.0.1:6001

As you can see, we used the add-node command, with the first argument being the address of the new node and the second argument being the address of a random existing node in the cluster.

The cluster has received a new master node, which you can verify by running the following command:-

./redis-cli -- cluster-check 127.0.0.1:6001

Let’s see how to re-shard the cluster.

Resharding the Cluster

Resharding involves moving hash slots from one group of nodes to another, and it's done with the redis-cli function, just like cluster setup. Simply type the below command to begin resharding: 

./redis-cli --cluster reshard 127.0.0.1:6001

You only need to specify a single node; redis-cli will discover the other nodes for you.

Currently, redis-cli can only reshard with administrator permission; you can't simply say "transfer 5% of slots from this node to the other" (but this is pretty trivial to implement). As a result, it all starts with a question. The first is how extensive a resharding you wish to perform. We can try resharding 1000 hash slots, which should already have a significant number of keys. Then redis-cli requires information on the resharding target, which is the node that will get the hash slots. We'll use the first master node, 127.0.0.1:6001, but we'll have to supply the instance's Node ID. This was already printed in a list by redis-cli, but if we need to, we can always use the following command to find the ID of a node:

./redis-cli -p 6001 cluster nodes | grep myself

You'll now be asked which nodes you wish to take those keys from. We'll simply type all to grab a few hash slots away from the other master nodes. The resharding process will then be completed. After the resharding is complete, run the following command to check the cluster's health:

./redis-cli --cluster check 127.0.0.1:6001

Adding a New Replica Node

There are two methods for adding a new Replica. The apparent solution is to rerun redis-cli with the — cluster-slave option, as seen below:

./redis-cli --cluster add-node 127.0.0.1:6007 127.0.0.1:6001 --cluster-slave

We are not indicating to which master we want to add the replica because the command line is identical to the one we used to add a new master. In this situation, redis-cli will add the new node as a replica of a random master from among those with the fewest replicas.

However, you can use the following command line to designate which master you want to target with your new replica:

./redis-cli --cluster add-node 127.0.0.1:6007 127.0.0.1:6001 --cluster-slave --cluster-master-id 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e

We can then assign the new replica to a specific master in this fashion.

To manually add a replica to a given master, add the new node as an empty master and then use the CLUSTER REPLICATE command to turn it into a replica. If the node was added as a slave, but you want to relocate it as a clone of a different master, this method also works.

To install a replica for the node 127.0.0.1:6006, which is now supplying hash slots in the range 11423–16383 and has the Node ID 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e, all we have to do is connect to the new node (which has already been added as an empty master) and issue the command:

./redis 127.0.0.1:6006> cluster replicate 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e

We now have a new replica for this set of hash slots, and all of the cluster's other nodes are aware of it (after a few seconds needed to update their config). The following command can be used to confirm:

./redis-cli -p 7000 cluster nodes | grep slave | grep 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e
f093c80dde814da99c5cf72a7dd01590792b783b 127.0.0.1:6006 slave 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e 0 1385543617702 3 connected

Removing Nodes from Existing Cluster

Simply use the redis-cli del-node command to remove a slave node:

./redis-cli --cluster del-node 127.0.0.1:6001 `<node-id>`

The first argument is simply a random cluster node, and the second argument is the ID of the node you want to remove.

A master node can be removed in the same way as a child node, except it must be empty to be removed. If the master is not empty, you must first reshard data to all of the other master nodes.

FAQs

What is Scaling in DBMS?

Scaling refers to the ability to increase a database system's capacity to accommodate larger queries and/or store more data without sacrificing performance.

What is Vertical Scaling?

Vertical scaling is the process of adding new resources to an existing system in order to fulfil a goal.

Which is Horizontal Scaling?

Horizontal scaling is when new server racks are added to an existing system to match increasing expectations.

Conclusion

In this article, we have extensively discussed Scaling in Dbms, types of scaling, and how to do horizontal scaling with the Redis cluster. If you want to learn more, check out our articles on IAM Security StandardData Warehousing ToolsThe Data Governance ChallengeData Virtualization Use Cases, and Encapsulation.

Do upvote our blog to help other ninjas grow.

Happy Coding!

Live masterclass