HiveQL到HBase

Question

I am using Hive 0.14 and Hbase 0.98.8 I would like to use HiveQL for accessing a HBase "table".

I created a table with a complex composite rowkey:

CREATE EXTERNAL TABLE db.hive_hbase (rowkey struct<p1:string, p2:string, p3:string>, column1 string, column2 string) 
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
COLLECTION ITEMS TERMINATED BY ';'
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = 
":key,cf:c1,cf:c2")
TBLPROPERTIES("hbase.table.name"="hbase_table");

The table is getting successfully created, but the HiveQL is taking forever:

SELECT * from db.hive_hbase WHERE rowkey.p1 = 'xyz';

Queries without using the rowkey are fine and also using the hbase shell with filters are working. I don't find anything in the logs, but I assume that there could be an issue with complex composite keys and performance.

Did anybody face the same issue? Hints to solve it? Other ideas, what I could try?

Thank you

Update 16.07.15: I changed the log4j properties to 'DEBUG' and found some interesting information:

It says:

2015-07-15 15:56:41,232 INFO  ppd.OpProcFactory    (OpProcFactory.java:logExpr(823)) - Pushdown Predicates of FIL For Alias : hive_hbase
2015-07-15 15:56:41,232 INFO  ppd.OpProcFactory (OpProcFactory.java:logExpr(826)) -     (rowkey.p1 = 'xyz')

But some lines later:

2015-07-15 15:56:41,430 DEBUG ppd.OpProcFactory (OpProcFactory.java:pushFilterToStorageHandler(1051)) - No pushdown possible for predicate:  (rowkey.p1 = 'xyz')

So my guess is: HiveQL over HBase does not do any predicate pushdown in Hbase but rather starts a MapReduce job.

Could there be a bug with the predicate pushdown?

Answer 1

I tried similar situation using Hive 0.13 and it works fine. I got the result. What version of hive are you working on?