Does Spark SQL (version 1.1.0) supports hive index?

Question

I used the spark SQL connect hive, basic query is OK, but was unable to create the index, will spark SQL support indexes?

DROP TABLE test_tabe;
CREATE  TABLE test_tabe
(
....
,area   STRING
...
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
STORED AS RCFILE
--STORED AS SEQUENCEFILE
;
CREATE INDEX idx_compact_index ON TABLE test_tabe (area) AS 'COMPACT' WITH DEFERRED REBUILD

#!!!!Execute always,no error,spark web page no corresponding job !!!!
ALTER INDEX idx_compact_index ON test_tabe REBUILD;

insert overwrite table test.test_tabe select * from test.new_mkt_browse_label_10m

Thanks.

Answer 1

Hive indexes are not supported in spark. They are less important because spark's in memory computation. By any chance have you run comparisons between indexed hive queries vs similar queries in spark? You can checkout more information on indexing not being implemented here