从hbase读取数据，工作人员是否需要从远程驱动程序获取分区数据？

Question

spark read data from hbase,such as //create rdd

val hBaseRDD = sc.newAPIHadoopRDD(conf, classOf[TableInputFormat], 
  classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable],
  classOf[org.apache.hadoop.hbase.client.Result])

for example, hBaseRDD has 5 partitions, now executor on worker get partition data to compute, they must get data from remote driver program? (not like read from hdfs, each worker as hadoop slave has the hdfs file replication)

Answer 1

Spark与HBase集成在一起，数据局部性原理与Hadoop map-reduce作业中的相同：spark将尝试将输入分区（hbase区域）分配给同一物理计算机上的工作器，因此无需远程驱动程序即可直接获取数据。