Apache Pig：在Pig中处理数据类型时面临问题
[英]Apache Pig: Facing issue while handling datatype in Pig
问题描述
I have been facing issue while handling the datatype for field qty and doing a SUM on the same field. 
我一直面临的问题，同时处理现场的数据类型qty ，并做了SUM在同一个领域。 Below is the code. 下面是代码。 I converted the qty to double but still getting the error mentioned below. 我将qty转换为double但仍然收到下面提到的错误。 Can someone please help me to understand this issue and if possible a solution? 有人可以帮我理解这个问题，如果可能的话，一个解决方案？
A_test1 = load'EXT_OO_IMP' USING PigStorage('\u0001') AS (it: chararray,loc: chararray,qty: chararray,scheddate: chararray,udc_cta_no: chararray,udc_imp_pack_qty: chararray,udc_imp_ready_dt: chararray,udc_imp_ref_no: chararray,udc_ord_sys_cd: chararray,udc_source: chararray,udc_sply_typ: chararray,udc_vend_pack_id: chararray,udc_purch_stg: chararray,srs_pack_flow_indicator_cd: chararray,it_type_cd: chararray,source_owner_cd: chararray,nks_id: chararray,alloc_replen_cd: chararray);
----- ext_oo_import: {it: chararray,loc: chararray,qty: chararray,scheddate: chararray,udc_cta_no: chararray,udc_imp_pack_qty: chararray,udc_imp_ready_dt: chararray,udc_imp_ref_no: chararray,udc_ord_sys_cd: chararray,udc_source: chararray,udc_sply_typ: chararray,udc_vend_pack_id: chararray,udc_purch_stg: chararray,srs_pack_flow_indicator_cd: chararray,it_type_cd: chararray,source_owner_cd: chararray,nks_id: chararray,alloc_replen_cd: chararray}
----- ############## ############## ##############
import_on_order =
FOREACH A_test1
GENERATE
loc,
it,
nks_id,
(double)(qty is NULL ? 0 : qty) as qty:double,
scheddate,
' ' AS order_source,
' ' AS chs_it_type_cd;
describe import_on_order;
----- import_on_order: {loc: chararray,it: chararray,nks_id: chararray,qty: int,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray}
grp_import_on_order = GROUP import_on_order BY (loc,it,nks_id,scheddate,order_source,chs_it_type_cd);
describe grp_import_on_order;
----- grp_import_on_order: {group: (loc: chararray,it: chararray,nks_id: chararray,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray),import_on_order: {(loc: chararray,it: chararray,nks_id: chararray,qty: int,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray)}}
------------------------------- STORE TO FILE ---------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------
work__idrp_import_on_order =
FOREACH grp_import_on_order
GENERATE group.loc AS loc,
group.it AS it,
group.nks_id AS nks_id,
SUM(import_on_order.qty) AS qty,
group.scheddate AS scheddate,
group.order_source AS order_source,
group.chs_it_type_cd AS chs_it_type_cd;
describe work__idrp_import_on_order;
----- work__idrp_import_on_order: {loc: chararray,it: chararray,nks_id: chararray,qty: int,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray}
import_on_order_rp =
FOREACH ext_oo_import
GENERATE
it AS chs_it,
loc AS chs_loc,
(qty is NULL ? 0 : qty) as qty:double,
scheddate AS current_due_dt,
' ' AS order_source,
'V' AS source_type_cd,
udc_sply_typ AS sply_typ,
udc_ord_sys_cd AS ord_sys_cd;
2019-01-31 09:03:30,819 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 0: Exception while executing (Name: grp_import_on_order: Local Rearrange[tuple]{tuple}(false) - scope-1095 Operator Key: scope-1095): org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing (Name: work__idrp_import_on_order: New For Each(false,false)[bag] - scope-1078 Operator Key: scope-1078): org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing (Name: Pre Combiner Local Rearrange[tuple]{Unknown} - scope-1097 Operator Key: scope-1097): org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing (Name: import_on_order: New For Each(false,false,false,false,false,false,false)[bag] - scope-977 Operator Key: scope-977): org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing (Name: ext_oo_import: New For Each(false,false,false,false,false)[bag] - scope-957 Operator Key: scope-95
1 个解决方案
解决方案1
0 2019-01-31 16:48:22
After looking at code, unable to get in first statement you are loading data and doing steps of transformation but again in last statament why you are transforming first data set again which is of string type and while working on this it giving exception. 在查看了代码之后，无法进入第一条语句，您正在加载数据并执行转换步骤，但是在最后一步中，为什么再次转换第一个字符串类型的数据集，并在处理该数据集时却给出异常。
import_on_order_rp = FOREACH ext_oo_import GENERATE it AS chs_it, loc AS chs_loc, (qty is NULL ? 0 : qty) as qty:double, scheddate AS current_due_dt, ' ' AS order_source, 'V' AS source_type_cd, udc_sply_typ AS sply_typ, udc_ord_sys_cd AS ord_sys_cd;
See if that is the correct case. 看看是否正确。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.