使用本机FS时的HBase批量加载

Question

I have a big list of HBase sql statements that I need to run against an existing instance of HBase database.

The SQL statements are in plain text, and the database is running on native file system (instead of HDFS). Thus, I do not have a Hadoop instance running.

I am currently running those SQL statements on an hbase shell, but the process is really slow. I know if Hadoop is running, I can assign mappers to do parallel inserts, but is there a faster way to do bulk load without using a Hadoop instance?

Answer 1

Since Hadoop (More specifically MR) is designed for faster processing of Bulk Data (Big data), it is apt for your requirement.

Create StoreFiles on HDFS and then use use CompleteBulkLoad methods, to load data from StoreFile to table.

Any specific reason for not using Hadoop Instance ?

Answer 2

HBase provides ImportTSV API for bulk upload

Here is an example of ImportTSv API

http://souravgulati.webs.com/apps/forums/topics/show/8617965-hbase-bulk-upload-using-importtsv-in-hbase-table

Here I tried uploading data from hdfs . You can try uploading data using ImportTSV from you local filesystem. If does not work then , run Hbase on top of HDFS