简体   繁体   中英

Hbase scan multiple versions in insertion order

I want to do a scan on hbase table for 10 versions. But, the result gives me data in latest to oldest order. I want to get it in the reverse order. Is there a way to do that?

Example :
If I put data in 'test' table in the following order :

put 'test','1','data:a','v0'
put 'test','1','data:a','v1'
put 'test','1','data:a','v2'

Scanning 3 versions gives me following order :

scan 'test',{VERSIONS=>3}
ROW COLUMN+CELL
1  column=data:a, timestamp=1537869886607, value=v2
1  column=data:a, timestamp=1537869884212, value=v1
1  column=data:a, timestamp=1537869881996, value=v0

I want to get the result in reverse order.

My full usecase is, to scan and put, so if I get the result in latest to oldest order, I will be writing in the reverse order, when I do put.
Code is here :

Scan scan = new Scan();
scan.setCacheBlocks(false);
scan.setCaching(10000);
scan.setMaxVersions(10);
ResultScanner scanner = tableGet.getScanner(scan);
for (Result result = scanner.next(); result != null; result = scanner.next()) {
  String row = new String(result.getRow());
  Put put = new Put(Bytes.toBytes(row));
  String key = "KEY" + ";" + row;
  for (Cell cell : result.rawCells()) {
    String family = Bytes.toString(CellUtil.cloneFamily(cell));
    String column = Bytes.toString(CellUtil.cloneQualifier(cell));
    byte[] value = CellUtil.cloneValue(cell);
    put.addColumn(family.getBytes(), column.getBytes(), value);
  }
  tablePut.put(put);
}

You can put the records with reverse order timestamps following 2 approaches:

  1. Put the row into HBase with explicit timestamp values.

Doing a put always creates a new version of a cell, at a certain timestamp. By default the system uses currentTimeMillis, but you can specify the timestamp (= the long integer) yourself, on a per-column level. This means you could assign a time in the past or the future, or use the long value for non-time purposes.

Initialize the timestamp value as:

long timestamp = Long.MAX_VALUE - System.currentTimeMillis()
Put put = new Put(Bytes.toBytes(rowKey), timestamp);
put.add(Bytes.toBytes(family), Bytes.toBytes(qualifier), Bytes.toBytes(value.toString()));
table.put(put);

Reference: https://hbase.apache.org/1.1/apidocs/org/apache/hadoop/hbase/client/Put.html#Put(byte[],%20long) https://www.ngdata.com/bending-time-in-hbase/

  1. Use Hashmap with Key as: "family_column" string (column family and column name concatenated with '|' or '_') and value LinkedList of values.

    HashMap<String, LinkedList> values = new HashMap<String, LinkedList>()

Insert the values as LinkedList with the Key. After the for loop, iterate the HashMap and for every element in HashMap, get the Value which is LinkedList and Reverse the LinkedList using:

Collections.reverse(list)

Now iterate through the reversed list and Put the elements in HBase.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM