shell 用于在文件中添加数据的脚本

Question

我在 linux 环境中有文件说 sample1.txt，并带有逗号分隔值：下面

"zc52","FS_SE",20-JUN-19,"P1C217.DB.ATC.COM  - 636","" 
 "p214","FS_MG",16-OCT-20,"","" 
 "x901","FS_TC",01-APR-21,"","" 
 "r750","FS_SP",06-JUL-21,"",

我需要具有 100 万条记录的以下格式的 output，我该怎么做。

{"id":"zc52","role":"FS_SE","created_time_stamp":"20-JUN-19, q_id":"P1C217.DB.ATC.COM - 636"}
{"id":"p214","role":"FS_MG","created_time_stamp":"16-OCT-20"}
{"id":"x901","role":"FS_TC","created_time_stamp":"01-APR-21"}
{"id":"r750","role":"FS_SP","created_time_stamp":"06-JUL-21"}

Answer 1

我会为此任务选择其他语言，因为使用 bash 准确解析引用的 CSV 有点困难。

使用 Ruby 真的很容易（没有外部依赖）。

csv2json.rb

#!/usr/bin/env ruby

require 'csv'
require 'json'

CSV.parse(ARGF, headers: ['id', 'role', 'created_time_stamp', 'q_id']) do |row|
  puts row.to_h.reject{|_,v| v.empty?}.to_json
end

note: This script can read a CSV from stdin, any number of file arguments, or both at the same time (example: cat file2.csv |./csv2json.rb file1.csv - file3.csv ). 输出到标准输出。

输入：

"zc52","FS_SE",20-JUN-19,"P1C217.DB.ATC.COM  - 636",""
"p214","FS_MG",16-OCT-20,"",""
"x901","FS_TC",01-APR-21,"",""
"r750","FS_SP",06-JUL-21,"",""

output：

{"id":"zc52","role":"FS_SE","created_time_stamp":"20-JUN-19","q_id":"P1C217.DB.ATC.COM  - 636"}
{"id":"p214","role":"FS_MG","created_time_stamp":"16-OCT-20"}
{"id":"x901","role":"FS_TC","created_time_stamp":"01-APR-21"}
{"id":"r750","role":"FS_SP","created_time_stamp":"06-JUL-21"}

备注：您要生成的不是有效的 JSON 而是一堆 JSON 对象。