使用awk / sed附加和替换

Question

I have this file:

2016,05,P,0002    ,CJGLOPSD8                                                    
00,BBF,BBDFTP999,051000100,GBP,   ,       -2705248.00                           
00,BBF,BBDFTP999,059999998,GBP,   ,       -3479679.38                           
00,BBF,BBDFTP999,061505141,GBP,   ,             -0.40                           
00,BBF,BBDFTP999,061505142,GBP,   ,        6207621.00                           
00,BBF,BBDFTP999,061505405,GBP,   ,             -0.16                           
00,BBF,BBDFTP999,061552000,GBP,   ,             -0.24                           
00,BBF,BBDFTP999,061559010,GBP,   ,             -0.44                           
00,BBF,BBDFTP999,062108021,GBP,   ,             -0.34                           
00,BBF,BBDFTP999,063502007,GBP,   ,             -0.28  

I want to programmatically (in unix, or informatica if possible) grab the first two fields in the top row, concatenate them, append them to the end of each line and remove that first row.

Like so:

00,BBF,BBDFTP999,051000100,GBP,,-2705248.00,201605                          
00,BBF,BBDFTP999,059999998,GBP,,-3479679.38,201605                           
00,BBF,BBDFTP999,061505141,GBP,,-0.40,201605                           
00,BBF,BBDFTP999,061505142,GBP,,6207621.00,201605                           
00,BBF,BBDFTP999,061505405,GBP,,-0.16,201605                           
00,BBF,BBDFTP999,061552000,GBP,,-0.24,201605                          
00,BBF,BBDFTP999,061559010,GBP,,-0.44,201605                         
00,BBF,BBDFTP999,062108021,GBP,,-0.34,201605                        
00,BBF,BBDFTP999,063502007,GBP,,-0.28,201605

This is my current attempt:

awk -vvar1=`cat OF\ OPSDOWN8.CSV | head -1 | cut -d',' -f1` -vvar2=`cat OF\ OPSDOWN8.CSV | head -1 | cut -d',' -f2` 'BEGIN {FS=OFS=","} {print $0, var 1var2}' OF\ OPSDOWN8.CSV> OF_OPSDOWN8.csv

Any pointers? I've tried looking around the forum but can only find answers to part of my question.

Thanks for your help.

Answer 1

Use this awk :

awk 'BEGIN{FS=OFS=","} NR==1{val=$1$2;next} {gsub(/ */,"");print $0,val}' file

Explanation:

• BEGIN{FS=OFS=","} - This block will set FS (Field Separator) and OFS (Output Field Separator) as , .
• NR==1 - Working with line number 1. Here, $1 and $2 denotes field number.
• print $0,val - Printing $0 (whole line) and stored value from val .

Answer 2

I would use the following awk command:

awk 'NR==1{d=$1$2;next}{$(NF+1)=d;gsub(/[[:space:]]/,"")}1' FS=, OFS=, file

Explanation:

• NR==1{d=$1$2;next} applies on line 1 and set's a variable d(ate) to the value of the first and the second field. The variable is being used when processing the remaining lines. next tells awk to go ahead with the next line right away without processing further instructions on this line.

• {$(NF+1)=d;gsub(/[[:space:]]/,"")}1 appends a new field to the line ( NF is the number of fields, assigning d to $(NF+1) effectively adds a field. gsub() is used to removing spaces. 1 at the end always evaluates to true and makes awk print the modified line.

• FS=, is a command line argument. It set's the input field delimiter to , .

• OFS=, is a command line argument. It set's the output field delimiter to , .

Output:

00,BBF,BBDFTP999,051000100,GBP,,-2705248.00,201605
00,BBF,BBDFTP999,059999998,GBP,,-3479679.38,201605
00,BBF,BBDFTP999,061505141,GBP,,-0.40,201605
00,BBF,BBDFTP999,061505142,GBP,,6207621.00,201605
00,BBF,BBDFTP999,061505405,GBP,,-0.16,201605
00,BBF,BBDFTP999,061552000,GBP,,-0.24,201605
00,BBF,BBDFTP999,061559010,GBP,,-0.44,201605
00,BBF,BBDFTP999,062108021,GBP,,-0.34,201605
00,BBF,BBDFTP999,063502007,GBP,,-0.28,201605

Answer 3

With sed :

sed '1{s/\([^,]*\),\([^,]*\),.*/\1\2/;h;d};/.*/G;s/\n/,/;s/ //g' file

in ERE mode :

sed -r '1{s/([^,]*),([^,]*),.*/\1\2/;h;d};/.*/G;s/\n/,/;s/ //g' file

Output :

00,BBF,BBDFTP999,051000100,GBP,,-2705248.00,201605
00,BBF,BBDFTP999,059999998,GBP,,-3479679.38,201605
00,BBF,BBDFTP999,061505141,GBP,,-0.40,201605
00,BBF,BBDFTP999,061505142,GBP,,6207621.00,201605
00,BBF,BBDFTP999,061505405,GBP,,-0.16,201605
00,BBF,BBDFTP999,061552000,GBP,,-0.24,201605
00,BBF,BBDFTP999,061559010,GBP,,-0.44,201605
00,BBF,BBDFTP999,062108021,GBP,,-0.34,201605
00,BBF,BBDFTP999,063502007,GBP,,-0.28,201605

Answer 4

This might work for you (GNU sed):

sed '1s/,//;1s/,.*//;1h;1d;s/ //g;G;s/\n/,/' file

For the first line only: remove the first comma, remove from the next comma to the end of the line, store the amended line in the hold space (HS) and then delete the current line (the d abruptly ends processing). For subsequent lines: remove all spaces, append the HS and replace the newline (from the G command) with a comma.

Or if you prefer:

sed '1{s/,//;s/,.*//;h;d};s/ //g;G;s/\n/,/' file

Answer 5

If you want to use Informatica for this, use two Source Qualifiers. Read the file twice - just one line in one SQ (filter out the rest) and in the second SQ read the whole file except the first line (skip header). Join the two on dummy port and you're done.