简体   繁体   English

猪HBASE失败; 储存CSV

[英]PIG HBASE FAILED ; store CSV

I'm beginning with the environment Hadoop, HBase, Pig; 我从Hadoop,HBase,Pig环境开始; I would storer a database on a Hbase table; 我将数据库存储在Hbase表上; here are my requeste: 这是我的要求:

hbase-> create to create my database. hbase-> create创建我的数据库。

I rebooted my CVS file on HDFS file hadoop dfs-copyFromLocal ~ /data.csv / user / hduser / location.csv 我在HDFS文件hadoop dfs-copyFromLocal〜/data.csv/user/hduser/location.csv上重新启动了CVS文件

I executed my requeste PIG 我执行了我要求的猪

raw_data = LOAD '/ user / hduser / data.csv' USING PigStorage (';') AS ( WMO: int, maturity: charArray, temperature: int, temps_present: int, direction_du_vent: int, force_du_vent: int, pressure: int); raw_data = LOAD'/ user / hduser / data.csv'使用PigStorage(';')AS(WMO:int,成熟度:charArray,温度:int,temps_present:int,direction_du_vent:int,force_du_vent:int,压力:int) ;

STORE INTO raw_data 'hbase :/ / TrafficFlowGrenoble' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage ( WMO: int maturity: charArray temperature: int temps_present: direction_du_vent int int force_du_vent int pressure: int '); 使用org.apache.pig.backend.hadoop.hbase.HBaseStorage存储到raw_data'hbase:/ // TrafficFlowGrenoble'(WMO:int成熟度:charArray温度:int temps_present:direction_du_vent int int force_du_vent int压力:int');

HERE IS MY ERRORS: 2013-07-22 22:30:31,870 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map REDUCE job (s) failed! 这是我的错误:2013-07-22 22:30:31,870 [main]错误org.apache.pig.tools.pigstats.PigStatsUtil-1地图减少任务失败! 2013-07-22 22:30:31,872 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: 2013-07-22 22:30:31,872 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats-脚本统计信息:

HadoopVersion PigVersion UserId StartedAt FinishedAt Features 1.1.2 0.11.1 hduser 2013-07-22 22:28:45 2013-07-22 22:30:31 UNKNOWN HadoopVersion PigVersion UserId StartedAt FinishedAt功能1.1.2 0.11.1 hduser 2013-07-22 22:28:45 2013-07-22 22:30:31未知

Failed! 失败了!

Failed Jobs: JobId Alias ​​Feature Message Outputs job_201307222219_0001 raw_data MAP_ONLY Message: Job failed! 作业失败:JobId别名功能消息输出job_201307222219_0001 raw_data MAP_ONLY消息:作业失败! Error - Failure JobCleanup Task, Task: task_201307222219_0001_m_000001 hbase :/ / TrafficFlowGrenoble, 错误-失败JobCleanup任务,任务:task_201307222219_0001_m_000001 hbase:/ / TrafficFlowGrenoble,

Input (s): Failed to read data from "/ user / hduser / data.csv" 输入(s):无法从“ / user / hduser / data.csv”读取数据

Output (s): Failed to Produce result in "hbase :/ / TrafficFlowGrenoble" 输出(一个或多个):无法在“ hbase:// TrafficFlowGrenoble”中产生结果

counters: Total records written: 0 Total bytes written: 0 Spillable Memory Manager spill count: 0 Total bags Proactively spilled: 0 Proactively spilled Total records: 0 计数器:写入的总记录数:0写入的总字节数:0 Spillable Memory Manager溢出计数:0袋子总数主动溢出:0主动溢出总数:0

vERSION : hadoop-1.1.2 pig-0.11.1 版本:hadoop-1.1.2 pig-0.11.1

export HBASE_HOME=/usr/lib/hbase 导出HBASE_HOME = / usr / lib / hbase

#export PIG_CLASSPATH=” ${HBASE_HOME}/bin/hbase classpath :$PIG_CLASSPATH” export HADOOP_CLASSPATH=” ${HBASE_HOME}/bin/hbase classpath :$HADOOP_CLASSPATH” #export PIG_CLASSPATH="$PIG_CLASSPATH:/usr/local/hbase/hbase-0.94.8.jar" export PIG_CLASSPATH="/usr/local/hbase/conf/hbase-site.xml:/usr/local/hbase/hbase-0.94.8.jar:/usr/local/hbase/lib/zookeeper-3.4.5.jar:/usr/local/hbase/lib/$ #export PIG_CLASSPATH=/home/nadir/Documents/pig-0.11.1.jar #export PIG_CLASSPATH =” ${HBASE_HOME}/bin/hbase classpath :$ PIG_CLASSPATH” export HADOOP_CLASSPATH =” ${HBASE_HOME}/bin/hbase classpath :$ HADOOP_CLASSPATH” #export PIG_CLASSPATH =“ $ PIG_CLASSPATH:/ usr / local / hbase / hbase-0.94.8.jar”导出PIG_CLASSPATH =“ / usr / local / hbase / conf / hbase-site.xml:/usr/local/hbase/hbase-0.94.8.jar:/ usr / local / hbase / lib /zookeeper-3.4.5.jar:/usr/local/hbase/lib/$ #export PIG_CLASSPATH = / home / nadir / Documents / pig-0.11.1.jar

如果上面的代码与您输入的文件完全一样,那么输入的文件是“ / user / hduser / location .csv”而不是data.csv,因此Pig找不到它

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM