使用 hive 查询进行数据解析

Question

I am building a pipeline through Azure data factory. Input dataset is a csv file with column delimiter and the output dataset is also a csv file column delimiter. The pipeline is designed with a HDinsight activity through hive query in the file with extension .hql. The hive query is as follows

set hive.exec.dynamic.partition.mode=nonstrict;

DROP TABLE IF EXISTS Table1; 
CREATE EXTERNAL TABLE Table1 (
  Number string, 
  Name string, 
  Address string
)
ROW FORMAT DELIMITED FIELDS  TERMINATED BY ','
LINES TERMINATED BY '\n'
STORED AS TEXTFILE 
LOCATION '/your/folder/location'

SELECT * FROM Table1;

Below is the file format

Number,Name,Address 
1,xyz,No 152,Chennai
2,abc,7th street,Chennai
3,wer,Chennai,Tamil Nadu

How do I data parse the column header with the data in the output dataset?

Answer 1

As per my understanding, Your question is related to to csv file. You are putting csv file at table location and it consist of header. If my understanding is correct, Please try below property in your table ddl. I hope this will help you.

tblproperties ("skip.header.line.count"="1");

Thanks, Manu