How to read date, time and Timestamp from hbase column

Question

I create following table in Hbase using Phoenix.

CREATE TABLE test_Table
( test_date date not null,
CONSTRAINT PK_test PRIMARY KEY (test_date)
);

Then insert one record into the same using following command.

upsert into test_Table(test_date) values('2013-11-30');

I am able to read string, int, float and double data type values from Hbase using. Hbase client API but not date type.

I am using following code to read all but not sure how to read date from Bytes.

    import org.apache.spark._
    import org.apache.spark.rdd._
    import org.apache.spark.sql.SQLContext
    import org.apache.spark.sql.DataFrame
    import org.apache.hadoop.conf.Configuration
    import org.apache.hadoop.fs.Path
    import org.apache.hadoop.hbase.HBaseConfiguration
    import org.apache.hadoop.hbase.spark.HBaseContext
    import org.apache.hadoop.hbase.client.Scan
    import org.apache.hadoop.hbase.util.Bytes
    import org.apache.spark.sql.types._
    import org.apache.hadoop.hbase.filter.PrefixFilter
    import org.apache.hadoop.hbase.{ TableName, HBaseConfiguration }
    import java.io.File
    import java.text.SimpleDateFormat

def scanHBaseTable(tableName: String, sqlContext: SQLContext): Unit = {

@transient val conf = getHbaseConfiguration();
@transient var scan = new Scan()
//scan.setAllowPartialResults(Constants.ALLOW_HBASE_PARTIAL_SCAN)
//scan.setCaching(Constants.HBASE_SCAN_CACHE)  

val hbaseContext = new HBaseContext(sqlContext.sparkContext, conf);
val hbaseRawRDD = hbaseContext.hbaseRDD(TableName.valueOf(tableName), scan)
hbaseRawRDD.foreach(v =>
  {
    println(Bytes.toString(v._1.get()))
    println((new SimpleDateFormat("yyyy-MM-dd").parse(Bytes.toString(v._1.get()))))
  })

println("Length: " + hbaseRawRDD.map(r => r._1.copyBytes()).collect().length);

}

Can someone provide me solution for the same?

Answer 1

The type of v is (ImmutableBytesWritable, Result) So you can get the date from Result object. You can use method result.getColumnLatestCell(family, qualifier).getTimestamp .

I don't know what families or qualifiers Phoenix is using, you can list all values inside the table and know the structure they are using. You can use method Result.getMap which returns Map[Family, Map[Qualifier, Map[CreateTime, Value]]]