Store Data to HBase
Preparation
If you use HBase to store the data, you need to install HBase correctly, and the details are as follows:
- Install HBase. Please install HBase when creating the cluster. For more information, please see Create Clusters. After the successful installation of HBase, the Master is deployed on BMR's Master node, and the Region Server is deployed on BMR's Core node.
- Log in to HBase Master node. That is to connect the BMR cluster's Master node. For more information, please see SSH Connection to Cluster.
- Log in to HBase Master node, and then log in to BMR's Core node through SSH: Take the private network address of Core node as the IP address for SSH connection, and obtain the port number from the message.
Read and Write Data
The steps of reading and writing data are as follows:
- Connect HBase. Log in to HBase Master node, and then execute the command hbase shell.
-
Create tables.
Example: If you want to create a table which is named' test' and contains a column family named' family', execute the command hbase(main):001:0> create 'test', 'family'.
- List all tables. Execute the command hbase(main):001:0> list.
-
Write the data.
Example: If you want to set ‘val1’ as the data of row ‘row1’ and column ‘family:col1’, execute the command hbase(main):001:0> put ‘test', ‘row1', ‘family:col1', and ‘val1’.
-
Read the data.
Example:
- If you want to read all the data of the row row1, execute the command hbase(main):001:0> get ‘test', and ‘row1’.
- If you want to read the data of row ‘row1’ and column ‘family:col1’, execute the command hbase(main):001:0> get ‘test', ‘row1', and ‘family:col1'.
Configure HBase
- Log in to the HBase Master node.
- Under the /etc/hbase/conf directory of Master node, find the configuration file hbase-sit.xml and the hbase-env.sh script for configuration of JVM and GC parameters, and then modify the parameters and save the configuration file.
-
After modifying the configuration file, restart the HBase Master node and the Region Server to put configurations into effect.
- Master’s restart command: su hbase -c ‘/opt/bmr/hbase/bin/hbase-daemon.sh --config /etc/hbase/conf restart master'
- Region Server’s restart command: su hbase -c ‘/opt/bmr/hbase/bin/hbase-daemon.sh --config /etc/hbase/conf restart regionserver’
Use Hive to Access HBase
Application Scenario
In BMR, you can combine HBase with Hive. When Hive is used to access HBase, Hive and HBase can be in the same cluster or the different clusters. It is generally recommended that you run HBase and Hive in different clusters to improve HBase's usage of cluster resources.
After connecting the Hive to HBase, you can create external tables in Hive to access the data stored in HBase.
Note:
You can only connect the Hive cluster to an individual HBase cluster.
Operating Steps
-
Connect Hive to HBase.
- Create the BMR cluster through the console, and then add the Hive application. For more information, please see Create Clusters.
- Use SSH to connect to the master node. For more information, please see SSH Connection to Cluster.
- Copy the
/etc/hosts
of the HBase cluster to the/etc/hosts
of the Hive cluster. - Execute the command
hive
to launch the Hive program. -
Connect the HBase client of the Hive cluster and the HBase cluster containing data. Execute the command
set hbase.zookeeper.quorum=hbase_cluster_master_ip
.Notes:
hbase_cluster_master_ip
is the master node private IP of the Hbase service cluster, and Hive and Hbase must be two clusters of the same user.- Skip this step if Hive and Hbase are in the same cluster.
-
Use Hive to access HBase data. During the running of the Hive prompt, the external table in the following example refers to the data stored on the HBase table named
hive_hbase
. After then, you can directly refer topagecounts_hbase
in the Hive statement to query and modify the data stored on the HBase cluster.CREATE EXTERNAL TABLE IF NOT EXISTS pagecounts_hbase (rowkey STRING, pageviews STRING, bytes STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES ('hbase.table.name' = 'hive_hbase');
select count(*) from pagecounts_hbase;
View HBase Logs
Go to the /var/log/hbase directory of HBase Master and Region server nodes to view the HBase logs.