Advanced Features in Cloud Bigtable

Delete an instance

When you use one of the Cloud Bigtable client libraries, you may remove an instance programmatically. To manually remove a Bigtable instance, follow these steps:

1. In the terminal, navigate to the list of Bigtable instances.
2. Click Delete instance after selecting the instance to be deleted. A confirmation dialog is shown.
   
   Src: https://cloud.google.com/bigtable/img/delete-instance-confirm.png 
3. Follow the prompts in the confirmation popup before clicking Delete. The instance is permanently removed.

Scaling clusters

Scaling a cluster is the process of adding or deleting nodes from a cluster in response to changes in the cluster's workload or data storage requirements. Scaling your Cloud Bigtable cluster depending on data such as CPU use might be beneficial. For example, if your cluster is under severe strain and has a high CPU utilization, you can add nodes to the cluster until the CPU utilization decreases. You may also save money by removing nodes from the cluster when they are not in use.

Scaling options

A Bigtable cluster can be scaled in the following ways:

• Autoscaling
• Manual node allocation
• Programmatically autoscaling

In most circumstances, you should utilize Bigtable's built-in autoscaling feature if you want automated scaling. When you enable this function, Bigtable monitors the cluster in real-time and automatically adjusts the number of nodes according to your preferences.

Limitations

Consider the following constraints before enabling autoscaling or configuring programmatic scaling for your Bigtable cluster.

Node availability

Node quotas apply regardless of whether a cluster uses manual node allocation or autoscaling.

Delay while nodes rebalance

It might take up to 20 minutes under load after adding nodes to a cluster before you see a meaningful increase in the cluster's performance. Consequently, if your workload includes short bursts of high activity, adding nodes to your cluster based on CPU load will not enhance performance since the short burst of activity will be finished by the time Bigtable rebalances your data.

To account for this delay, add nodes to your cluster before increasing its demand, either programmatically or using the Google Cloud app. This strategy allows Bigtable to redistribute your data among the new nodes before the burden grows. Change the number of nodes in clusters that employ manual node allocation.

Latency increases caused by scaling down too quickly

When scaling down the number of nodes in a cluster, try not to reduce the cluster size by more than 10% in a 10-minute period. Scaling down too rapidly might result in performance issues, such as increased latency if the cluster's surviving nodes become suddenly overloaded.

Schema design issues

If your table's schema design is flawed, adding nodes to your Bigtable cluster may not increase performance. For example, if you have many reads or writes to a single row in your table, they will all travel to the same node in your cluster; hence, more nodes will not enhance speed. In contrast, increasing nodes will often enhance speed if your table's reads and writes are uniformly spread across rows.

How to scale Bigtable programmatically

You may wish to create your application to scale your Bigtable cluster in some situations. This section discusses how to scale a cluster programmatically and includes a code sample to get you started. It also discusses certain limits to be aware of before using programmatic scaling.

Bigtable's Cloud Monitoring API offers several metrics. You may monitor these metrics for your cluster programmatically, then use one of the Bigtable client libraries or the Google Cloud CLI to add or delete nodes according to the metrics. After resizing your cluster, you may track its performance via the console, a Cloud Monitoring custom dashboard, or programmatically.

Monitoring API metrics

The Monitoring API provides a number of metrics for monitoring the present condition of your cluster. The following are some of the most relevant metrics for programmatic scaling:

• bigtable.googleapis.com/cluster/cpu_load: Cluster's CPU load.
• bigtable.googleapis.com/cluster/node_count: Number of nodes in the cluster.
• bigtable.googleapis.com/cluster/storage_utilization: Storage used as a fraction of total storage capacity.
• bigtable.googleapis.com/server/latencies: Distribution of server request latencies for a table.

Sample code

You can use one of the following sample tools as a starting point for your own programmatic scaling tool:

• Java sample tool
• Python sample tool

When a node's CPU load exceeds a certain threshold, the sample tools join it to a Bigtable cluster. Similarly, the sample tools delete nodes from a Bigtable cluster when their CPU load falls below a certain threshold. To execute the sample tools, follow the instructions on GitHub for each example.

To acquire information on the CPU load on the cluster, the sample tools apply the following code:

Timestamp now = timeXMinutesAgo(0);
Timestamp fiveMinutesAgo = timeXMinutesAgo(5);
TimeInterval interval =
    TimeInterval.newBuilder().setStartTime(fiveMinutesAgo).setEndTime(now).build();
String filter = "metric.type=\"" + CPU_METRIC + "\"";
ListTimeSeriesPagedResponse response =
    metricServiceClient.listTimeSeries(projectName, filter, interval, TimeSeriesView.FULL);
return response.getPage().getValues().iterator().next().getPointsList().get(0);

The sample tools employ the Bigtable client library to scale the cluster based on the CPU 

load:

double latestValue = getLatestValue().getValue().getDoubleValue();
if (latestValue < CPU_PERCENT_TO_DOWNSCALE) {
  int clusterSize = clusterUtility.getClusterNodeCount(clusterId, zoneId);
  if (clusterSize > MIN_NODE_COUNT) {
    clusterUtility.setClusterSize(clusterId, zoneId,
      Math.max(clusterSize - SIZE_CHANGE_STEP, MIN_NODE_COUNT));
  }
} else if (latestValue > CPU_PERCENT_TO_UPSCALE) {
  int clusterSize = clusterUtility.getClusterNodeCount(clusterId, zoneId);
  if (clusterSize <= MAX_NODE_COUNT) {
    clusterUtility.setClusterSize(clusterId, zoneId,
      Math.min(clusterSize + SIZE_CHANGE_STEP, MAX_NODE_COUNT));
  }
}

Autoscaling

The number of nodes in the cluster remains fixed with manual node allocation until you modify it. Bigtable continually monitors the cluster and automatically modifies the number of nodes in the cluster when autoscaling is enabled. Autoscaling is available in all Bigtable regions and works on HDD and SSD clusters.

Advantages of Autoscaling

The following are some of the advantages of autoscaling:

Costs

Because Bigtable decreases the number of nodes in your cluster whenever possible, autoscaling can help you save money. This might assist you in avoiding over-provisioning.

Performance

Bigtable's autoscaling feature allows it to automatically add nodes to a cluster when a workload changes or data storage requirements rise. This contributes to the achievement of workload performance goals by ensuring that the cluster has enough nodes to fulfill the required CPU usage and storage needs.

Automation

Autoscaling simplifies management. You don't need to manually monitor and scale the cluster size or develop an application to do so because the Bigtable service does these activities for you.

How autoscaling works

The technique of automatically scaling or altering the size of a cluster by adding or deleting nodes is known as autoscaling. Bigtable automatically changes the size of your cluster when you enable autoscaling. When your cluster's workload or storage requirements change, Bigtable either scales up, adds nodes to the cluster, or scales down, eliminating nodes from the cluster.

Bigtable autoscaling calculates the number of nodes needed depending on the following parameters:

• CPU utilization target
• Storage utilization target
• Minimum number of nodes
• Maximum number of nodes

Each scaling dimension provides a suggested node count, and Bigtable chooses the highest one by default. Bigtable grows the cluster to 12 nodes if, for example, your cluster requires 10 nodes to reach your storage usage objective but 12 to satisfy your CPU utilization target.

Bigtable continually optimizes storage, rebalancing data between nodes as the number of nodes varies to guarantee that traffic is distributed relatively and no node is overwhelmed.

Bigtable automatically rebalances the nodes in your cluster after scaling it up for maximum performance. While scaling and rebalancing occur, all requests continue to reach the cluster. 

Requests may have an excessive delay or fail if a cluster has reached its maximum number of nodes and the CPU usage target has been surpassed. Write requests will fail if a cluster has reached its maximum number of nodes and the storage utilization limit has been exceeded.

To avoid any influence on latency, nodes are eliminated at a slower pace when a cluster is scaled-down than when it is scaled up.

Frequently Asked Questions

What is the difference between cloud Datastore and Cloud Bigtable?

BigTable is designed for big data volumes and analytics, whereas Datastore is designed to offer high-value transactional data to applications.

How is data stored in Bigtable?

Bigtable is a row-oriented database, which means that all data for a single row is saved simultaneously, first by column family, then by column. Data is kept in reversed-timestamp order, so getting the most recent value is simple and quick while getting the oldest value is difficult.

What is Rowkey in Bigtable?

You may give each dataset a unique row key prefix so that Bigtable stores the linked data in a continuous range of rows that you can then query by row key prefix.

Conclusion

In this article, we have extensively discussed instances in Cloud Bigtable. Our discussion mainly focused on advanced features of the Cloud Bigtable, such as modifying and deleting instances in Cloud Bigtable. We also learned about scaling clusters and Autoscaling.

We hope this blog has helped you enhance your Google cloud knowledge. To learn more about Google cloud concepts, refer to our articles on All about GCP Certifications: Google Cloud Platform | Coding Ninjas Blog.  

Refer to our guided paths on the Coding Ninjas Studio platform to learn more about DSA, DBMS, Competitive Programming, Python, Java, JavaScript, etc. 

Refer to the links problems, top 100 SQL problems, resources, and mock tests to enhance your knowledge.

For placement preparations, visit interview experiences and interview bundle.

Do upvote our blog to help other ninjas grow. Happy Coding!