在终端的每一行的开头插入文件名中更改的字符串

Question

I have zero experience with bash so I'm struggling with the syntax - I gave up attempting in python as I thought that it might be easier. I would like to extract part of the filename (before the .xyz extension, and after the prefix), insert it on every line (starting from the third line) and pipe the output to a new file. I would also like to do this for multiple files, where the string changes.

My input files are as follows:

blahblah-0.xyz
blahblah-1.xyz
blahblah-2xyz

So far I know that I can do:

sed '3,$ s/^/500 /' file-500.xyz > output

and this will insert the string on every line. But I don't want to do this 100 times for each directory! I also tried the following from here: awk parse filename and add result to the end of each line :

 for filename in ratio*; do 
   num=$(echo $filename | grep -Eo '[^ratio_distances]+\.xyz' | cut -d. -f1)
   sed -i "s/\^/\t$num" $filename
 done

Just to add, this is just being performed in the standard mac terminal, as I've had errors crop up in regards to the 'sed -i' command.

EDIT:

I got it to work in python, but I'd still be interested to know the bash commands. Python code should any one else be after the same thing:

import sys
import os
import glob

list_of_files = glob.glob("./blah-blah*.xyz")
for file in list of files:
    for i in range (0, 80):
        P = 10*i
        if str(P) in file:
            with open(file, 'r') as infile:
                lines = infile.readlines()
                lines[:]=lines[2:]
             lines = [str(P)+' '+line for line in lines]
             with open(file.replace('blahblah','output'),'w') as outfile:
                 outfile.writelines(lines)
             infile.close()
             outfile.close()

Thanks very much for any insight, Anna

Answer 1

Assuming you can just prefix the old file names with "new_" to create the new file names:

awk '
    FNR==1 { pfx = FILENAME; sub(/.*\./,"",pfx) }
    FNR>=3 { $0 = pfx $0 }
    { print > ("new_"FILENAME) }
' ratio*

Answer 2

You can use bash's parameter expansion to extract the number from the file name. The -i is not supported in Mac's sed, so you have to use a temp file:

#! /bin/bash
for filename in ratio* ; do
    num=${filename#ratio_distances-}  # Remove from the left.
    num=${num%.xyz}                   # Remove from the right.
    sed "3,\$ s/^/$num /" "$filename" > new"$num"
    mv new"$num" "$filename"          # Replace the original with the tempfile.
done

Answer 3

#!/bin/bash
PrefixFile="blahblah"
awk -v "Prefix=${PrefixFile}" '
   # At each new file starting
   FNR == 1 {
      # take number from current file name
      gsub( "^" Prefix "-|[.]xyz$", "", Index = FILENAME)
      }
   # at each line (so for every files)
   {
      # print the index file (current) followed by original line
      # to the (corresponding) filename.New
      print Index $0 > ( FILENAME ".New" )
   }
   ' ${PrefixFile}*.xyz

• Using awk, using all files at once from shell expansion
  • assume prefix is without the - (easily adapteable if not)
• ouput culd be any other file except itself (modification of aw could also change the name at the end but better in bash itself)

Thanks to @EdMorton for extra info about behaviour of awk about file redirection