Unable to create Hive table from spark

Question

I am trying to create hive table with Spark. I am getting the below error -

 +- TungstenAggregate(key=[rpt_prd#244,country_code#240,product_code#242], functions=[(count(1),mode=Partial,isDistinct=false)], output=[rpt_prd#244,co
untry_code#240,product_code#242,count#832L])
                     +- HiveTableScan [rpt_prd#244,country_code#240,product_code#242], MetastoreRelation gfrrtnsd_standardization, pln_arrg_dim, None, [(country_code#24
0 = HK)]

org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree:
TungstenExchange hashpartitioning(rpt_prd#236,200), None
+- Sort [rpt_prd#236 ASC], true, 0
   +- ConvertToUnsafe
      +- Exchange rangepartitioning(rpt_prd#236 ASC,200), None
         +- ConvertToSafe
            +- TungstenAggregate(key=[rpt_prd#244,country_code#240,product_code#242], functions=[(count(1),mode=Final,isDistinct=false)], output=[rpt_prd#236,Dim_countr
y_code#237,Dim_product_code#238,Dim_recordCount#239L])
               +- TungstenExchange hashpartitioning(rpt_prd#244,country_code#240,product_code#242,200), None
                  +- TungstenAggregate(key=[rpt_prd#244,country_code#240,product_code#242], functions=[(count(1),mode=Partial,isDistinct=false)], output=[rpt_prd#244,co
untry_code#240,product_c`enter code here`ode#242,count#832L])
                     +- HiveTableScan [rpt_prd#244,country_code#240,product_code#242], MetastoreRelation gfrrtnsd_standardization, pln_arrg_dim, None, [(country_code#24
0 = HK)]

Please assist to create a hive table from spark.

Answer 1

Mostly tree node exceptions will occur in this following secnario.

Let say you have a dataframe df1 and created a new dataframe df2 by applying some transformations and latter joining df1 with df2 is causing tree treenode exception in recent version.

So the Solutions might be

you have to create dataframe df1 and created a new dataframe df2 same as df1 and apply transformation on df2 and create df3 then do join operation with df3 and df1.