Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
HBase Troubleshooting
2.1.
Table Creation Fails after Installing LZO
2.2.
Thrift Server Crashes after Receiving Invalid Data
2.3.
Master server initializes, but region servers do not initialize
2.4.
Couldn’t find my address: ABC in the list of Zookeeper quorum servers
2.5.
Created Root Directory for HBase through Hadoop DFS
2.6.
Zookeeper session expired events
3.
FAQs
4.
Key Takeaways
Last Updated: Mar 27, 2024

HBase Troubleshooting

Author Sanjana Yadav
0 upvote
Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

HBase is a non-relational column-oriented database management system that works on top of the Hadoop Distributed File System (HDFS). HBase provides a fault-tolerant method of storing sparse data sets, common in many big data applications. It's suitable for real-time data processing or random read/write access to enormous amounts of data.

HBase systems are intended to scale linearly. It is made up of regular tables with rows and columns, similar to a typical database. Each table must have a primary key declared, and all access attempts to HBase tables must utilize this primary key.

While working with HBase, a variety of issues may arise.

HBase Troubleshooting

Table Creation Fails after Installing LZO

You won't be able to construct a table with LZO compression until you restart the RegionServer if you install LZO after starting it.

Cause
When the RegionServer begins, CompressionTest is conducted, and the results are cached. It refers to such results when you try to create a table with a certain type of compression. Because you installed LZO after starting the RegionServer, the create fails because the cached results predate LZO.

Solution
The RegionServer should be restarted. LZO table creation will now be successful.

Thrift Server Crashes after Receiving Invalid Data

A Thrift server may crash if it gets a substantial quantity of incorrect data as a result of a buffer overrun.

Cause
The Thrift server uses memory to validate the data it receives. It may need to allocate more memory than is available if it gets a significant volume of invalid data. This is because of a constraint in the Thrift library.

Solution

  • Use the framed and compact transport protocols to avoid any problems caused by buffer overruns. 
  • Because they may need changes to your client code, these protocols are disabled by default.
  •  hbase.regionserver.thrift.framed and hbase.regionserver.thrift.compact are two options to include in your hbase-site.xml. 
  • As seen in the XML below, set each of these to true. You may also use the hbase.regionserver.thrift.framed.max_frame_size_in_mb option to specify the maximum frame size.
<property> 
  <name>hbase.regionserver.thrift.framed</name> 
  <value>true</value> 
</property> 
<property> 
  <name>hbase.regionserver.thrift.framed.max_frame_size_in_mb</name> 
  <value>2</value> 
</property> 
<property> 
  <name>hbase.regionserver.thrift.compact</name> 
  <value>true</value> 
</property>

Master server initializes, but region servers do not initialize

Communication between master and region servers takes place mainly through their IP addresses. Master will listen to see whether any region servers are running or have the IP address 127.0.0.1.

Cause
In dual communication between region servers and master servers, region servers regularly tell the Master server about their IP addresses, which are 127.0.0.1.

Solution
Remove the master server name node from localhost, which is present in the hosts file.
Go to the Host file location /etc/hosts and after opening /etc./hosts,
Go to,
127.0.0.1 fully.qualified.regionservernameregionservernamelocalhost.localdomain localhost
: : 1 localhost3.localdomain3 localdomain3

Now modify the above configuration as:
127.0.0.1 localhost.localdomainlocalhost
: : 1 localhost3.localdomain3 localdomain3

Couldn’t find my address: ABC in the list of Zookeeper quorum servers

Causes

  1. The ZooKeeper server will raise an error with the suffix .abc in the server's name and will also fail to start.
  2. In addition, HBase tries to launch a ZooKeeper server on some machines, but the machines cannot locate the quorum configuration.

Solution

  1. Its solution is to substitute the hostname with a hostname that is presented in the error message.
  2. The following parameters can be configured in HBase-site.xml, but only if we assume that we have a DNS server.
  • HBase.zookeeper.dns.interface
  • HBase.zookeeper.dns.nameserver

Created Root Directory for HBase through Hadoop DFS

As we must execute the HBase migrations script. When the HBase migrations script is executed, it responds as if there are no files in the root directory.

Cause
Upon creating a new directory for HBase using the Hadoop Distributed file system. There are two possibilities:

  1. The root directory does not exist.
  2. Previously, just the previous running instance of HBase was initialized.
     

Solution
Ascertain that the HBase root directory does not currently exist or that it has been initialized by a previous execution of the HBase instance.

  1. Using Hadoop dfs, remove the HBase root directory.
  2. HBase creates and initializes the directory on its own.

Zookeeper session expired events

Cause
Exceptions cause HMaster or HRegion servers to stop down.

Furthermore, by inspecting logs, we may learn about the thrown exceptions.

The following exception is thrown as a result of the Zookeeper expired event:

WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x278bd16a96000f to sun.nio.ch.SelectionKeyImpl@355811ec
java.io.IOException: TIMED OUT
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:906)
WARN org.apache.hadoop.hbase.util.Sleeper: We slept 79410ms, ten times longer than scheduled: 5000
INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server hostname/IP:PORT
INFO org.apache.zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/IP:PORT remote=hostname/IP:PORT]
INFO org.apache.zookeeper.ClientCnxn: Server connection successful
WARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x278bd16a96000d to sun.nio.ch.SelectionKeyImpl@3544d65e
java.io.IOException: Session Expired
at org.apache.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java:589)
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:709)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: ZooKeeper session expired

Solution

  1. By default, the RAM size is 1 GB. As a result, we've kept RAM capacity over 1 GB for long-running imports. 
  2. Zookeeper's session timeout must be increased. To increase the session time out of Zookeeper, we must edit the following value in "hbase-site.xml," located in the hbase /conf folder path.
  3. The session timeout is set to 60 seconds by default. So,we may alter it to 120 seconds.
<property>
<name> zookeeper.session.timeout </name>
<value>1200000</value>
</property>
<property>
<name> hbase.zookeeper.property.tickTime </name>
<value>6000</value>
</property>

 

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

FAQs

  1. What type of database is HBase?
    HBase is a non-relational column-oriented database management system that works on top of the Hadoop Distributed File System (HDFS). HBase provides a fault-tolerant method of storing sparse data sets, which are common in many big data applications.
     
  2. What is the purpose of HBase?
    HBase's purpose is to store and analyze massive volumes of data, notably enormous amounts of data consisting of thousands of rows and columns, utilizing just ordinary hardware configurations.
     
  3. How many masters are possible in HBase?
    A cluster in Hbase is made up of one Master and three or more Region Servers.
     
  4. What are the issues with HBase?
    In HBase, no cross-database operations or joining operations are available.
    HBase would require a new design if we wanted to convert data from RDBMS external sources to HBase servers.
    HBase is quite difficult to query.
     
  5. Where are HBase logs?
    You may access the most recent HBase logs by connecting to the master node through SSH and browsing to the /var/log/hbase directory. Unless you enable logging to Amazon S3 when you begin the cluster, these logs are not available when the cluster is stopped.

Key Takeaways

Cheers if you reached here! In this blog, we learned about Troubleshooting in HBase.

We saw solutions to a variety of issues that commonly arise during HBase Troubleshooting.

On the other hand, learning never ceases, and there is always more to learn. So, keep learning and keep growing, ninjas!

Check out the Top 100 SQL Problems to get hands-on experience with frequently asked interview questions and land your dream job.

Good luck with your preparation!

Previous article
HBase Client API
Next article
HBase Performance Tuning
Live masterclass