如何使用awk读取每n个字符而不是每行的文件？

Question

This is the content of file.txt :

hello bro
my nam§
is Jhon Does

The file could also contain non-printable characters (for example \\x00, or \\x02) , and, as you can see, the lenght of the lines are not the same.

Then I want to read it each each 5 characters without having into a count line breaks. I thought in something like this using awk:

awk -v RS='' '{
  s=s $0;
}END{
  n=length(s);

  for(x=1; x<n; x=x+5){
    # Here I will put some calcs and stuff

    i++;
    print "line " i ": #" substr(s,x,5) "#"
  }
}' file.txt

The output is the following:

line 1: #hello#
line 2: # bro
#
line 3: #my na#
line 4: #m§
is#
line 5: # Jhon#
line 6: # Does#

It works perfectly, but the input file will be very large, so the performance is important.

In short, I'm looking for something like this:

awk -v RS='.{5}' '{ # Here I will put some calcs and stuff }'

But it doesn't works.

Another alternative that works ok:

xxd -ps mifile.txt | tr -d '\n' | fold -w 10 | awk '{print "23" $0 "230a"}' | xxd -ps -r

Do you have any idea or alternative? Thank you.

Answer 1

You can use perl and binmode assuming you are using normal characters.

use strict;
use warnings;

open my $fh, '<', 'test'; 
#open the file.
binmode $fh;
# Set to binary mode
$/ = \5;
#Read a record as 5 bytes

while(<$fh>){
#Read records
        print "$_#"
        #Do whatever calculations you want here
}

For extended character sets you can use UTF8 and read every 5 characters instead of bytes.

use strict;
use warnings;

open my $fh, '<:utf8', 'test';
#open file in utf8.
binmode(STDOUT, ":utf8");
# Set stdout to utf8 as well

while ((read($fh, my $data, 5)) != 0){
#Read 5 characters into variable data
    print "$data#";
    #Do whatever you want with data here
}

Answer 2

If you are okay with Python , You may try this

f = open('filename', 'r+')
w = f.read(5)
while(w != ''):
        print w;
        w = f.read(5);
f.close()

Answer 3

So you asked How to read a file each n characters instead of each line using awk .

Solution :

If you have a modern gawk implementation use FPAT

Normally, when using FS, gawk defines the fields as the parts of the record that occur in between each field separator. In other words, FS defines what a field is not, instead of what a field is. However, there are times when you really want to define the fields by what they are , and not by what they are not.

Code:

gawk 'BEGIN{FS="\n";RS="";FPAT=".{,5}"}
            {for (i=1;i<=NF;i++){
               printf("$%d = <%s>\n", i, $i)}
            }' file

Check the demo

Answer 4

I'm not sure I understand what you want but this outputs the same as the script in your question that you say works perfectly so hopefully this is it:

$ awk -v RS='.{5}' 'RT!=""{ print "line", NR ": #" RT "#" }' file
line 1: #hello#
line 2: # bro
#
line 3: #my na#
line 4: #m§
is#
line 5: # Jhon#
line 6: # Does#

The above uses GNU awk for multi-char RS and RT.