Hbase排序效率

Question

In my Hbasetable I have Employee name "Simon" at row-100 and at row-4000 I have another employee with same name "Simon". Now I want to get all employees with name "Simon" from my Employee table.The row-key is the SSN of each employee.

My question is,if i fire a query to get all employees with the name "Simon" .How is the search efficient in Hbase. Because the first name "simon" is in row 100 and second "simon" name is in 4000.To get employess with name "simon" hbase has to traverse all the table to find out this name .How will be the search efficient as we are doing a full table scan in this scenario?

Answer 1

If you have to do a full table scan - which you do - that's not going to be a great solution. In fact, if you have a very large number of rows, it's going to be a terrible solution.

What most relational database management systems (or "SQL databases") do to solve this problem is create indexes . Since you're using a "NoSQL database," it won't create indexes for you automatically.

Let's look at how to create indexes manually so particular types of queries are accommodated efficiently.

---

Suppose you have a collection of entities S where each entity E in S has a unique key K(E) and an attribute value V(E) . Further suppose your entities are in an HBase table, one per row, with K(E) as the row key for each entity E .

An index of S with respect to V is another table that typically comes in one of three forms.

Index Form 1

Suppose that V(E) is also unique for each entity E . Then the index of S with respect to V is a table with one entity per row, where the table has row key V(E) and a column containing K(E) .

To look up an entity E by V(E) , simply go to that row to look up K(E) .

If your attribute values V(E) are unique, use this approach.

Think a table of Employee entities, where each employee has a unique EmployeeID within the company, K(E) . The main Employee table could use the unique EmployeeID as the row key, and the Employee_SSN_Index could use the employee SSN number V(E) (which is also unique). This provide a fast lookup of employees by their SSN numbers.

Index Form 2

Suppose that V(E) is potentially not unique for each entity E ; that is, there may be duplicates. Then the index of S with respect to V is a table with one entity per row, where the table has a row key of V(E) ++ K(E) .

To look up all the entities E with V(E) , simply do a prefix scan of the rows that start with the V(E) .

There is a variant for the case when the length of V(E) is not fixed with and it may be impossible to distinguish the point at which V(E) ends and K(E) begins. A separator may be placed between V(E) and K(E) in the row key. For example V(E) ++ "|" ++ K(E) V(E) ++ "|" ++ K(E) . In this case, the prefix to scan is V(E) ++ "|" .

A Employee_Department_Index table could use the DepartmentID an employee works in as the attribute value V(E) .

Index Form 3

Suppose that V(E) is potentially not unique for each entity E ; that is, there may be duplicates. Then the index of S with respect to V is a table with a set of entities per row, where the table has a row key of V(E) and a column family F with qualifier K(E) . That is, the entities are grouped by the attribute value into rows.

To look up all the entities E with V(E) , grab the row V(E) requesting all columns in the column family F .

This approach should really be kept to the case where the number of entities in each row of the index is small.