[英]HBase: How can I estimate the size of a HBase table?
我有多個HBase表,如何估計在Java中使用的表的大致大小?
一種方法是通常必須在/hbase
文件夾下的所有表信息下使用Java客戶端訪問hdfs。 將存在。
您可以使用hadoop fs -du -h **path to hbase**/hbase
在/ hbase下,每個表又占用一個文件夾...
hadoop fs -ls -R **path to hbase**/hbase
hadoop fs -du -h **path to hbase**/hbase/tablename
同樣,您可以通過在hbase根目錄下傳遞每個表路徑來使用Java hdfs客戶端,如下所示:檢查getSizeOfPaths
和getSizeOfDirectory
方法
public class HdfsUtil {
/**
* Estimates the number of splits by taking the size of the paths and dividing by the splitSize.
*
* @param paths
* @param configuration
* @param splitSize
* @return
* @throws IOException
*/
public static long getNumOfSplitsForInputs(Path[] paths, Configuration configuration, long splitSize) throws IOException
{
long size = getSizeOfPaths(paths, configuration);
long splits = (int) Math.ceil( size / (splitSize)) ;
return splits;
}
public static long getSizeOfPaths(Path[] paths, Configuration configuration) throws IOException
{
long totalSize = 0L;
for(Path path: paths)
{
totalSize += getSizeOfDirectory(path, configuration);
}
return totalSize;
}
// here you can give hbase path folder which was described through shell
public static long getSizeOfDirectory(Path path, Configuration configuration) throws IOException {
//Get the file size of the unannotated Edges
FileSystem fileSystem = FileSystem.get(configuration);
long size = fileSystem.getContentSummary(path).getLength();
/**static String byteCountToDisplaySize(BigInteger size)
Returns a human-readable version of the file size, where the input represents a specific number of bytes.**/
System.out.println(FileUtils.byteCountToDisplaySize(size))
return size;
}
}
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.