Can you use sed or perl -pe to remove a part from a binary file?

Question

I am trying to remove parts from a binary file which are between the ANSI strings "stringstart" and "stringend". Is it possible to do this with sed or perl -pe?

I am thinking about some Regex solution but I don't know how to write it or how well regex works with binary files.

Answer 1

sed is designed for handling text files rather than binary, though these days, the distinction is generally less significant than it once was. The biggest problem is that text files do not contain zero bytes (bytes with value 0) and binary files do, and many C string processing functions stop at the first zero byte. sed also reads 'lines' marked by newline characters. Binary files can end up with long lines as a result. Finally, there's no guarantee about the relative placement of the string start and end markers relative to newlines. All of these characteristics make sed less suitable for this job than Perl is.

In Perl, I'd be sorely tempted to slurp the file into memory, use an appropriate regex to zap the data from the memory image, and then write the result back to an appropriate location.

perl -e 'local($/); $data = <>; $data =~ s/stringstart(.*?)stringend//gms; print $data'

---

Now tested - test data created using:

#!/usr/bin/env perl
use strict;
use warnings;

sub full_set
{
    foreach my $i (0..255) { printf "%c", $i; }
}
sub random_set
{
    my($n) = @_;
    foreach my $i (0..$n) { printf "%c", int(rand(255)); }
}

full_set;
random_set(1024);
printf("stringstart");
full_set;
random_set(512);
full_set;
printf("stringend");
random_set(256);

The script deletes 1045 characters from the input - which corresponds to 'stringstart', 'stringend' (20) + 2 * 256 + 513 (since random_set(512) prints 513 characters).

Note that the main script will read all files into memory at once. If you want it to process one file at a time, you'll have to work a little harder; it probably ceases to be a one-liner.

Answer 2

另一种方法：

perl -pi -we'BEGIN{$/="stringend"} chomp and s/stringstart.*//s' your_binary_file

Answer 3

You can do a regular expression that kills all characters not defined after the ^ inside of [] . For example

cp /bin/ls ./binfile
file binfile
binfile: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped

Do the perl pie on it:

perl -pi -e 's/[^[a-zA-Z0-9_+\n]//g' binfile

Then look at the binary file afterwards:

file binfile
binfile: ASCII text, with very long lines

You'll obviously have to add more to that command, as it'll get rid of several other would-be valid characters. But this should get you started.