How to write to dump file in HEX after reading in python 2.x

Question

As per the title I need help with writing to a specific byte in a dump file. So far I'm able to read 512 byte with the following code :

sectorcount = 0;
bytecount= 0;
with open('a2.dump', 'rb') as f:
    for chunk in iter(lambda: f.read(16), b''):
        #16 bytes per chunk aka 32 characters
        item = chunk.encode('hex')
        #to filter display output so it shows 2 character per array element
        filtered_item= [item[i:i+2] for i in range(0, len(item), 2)]
        #to display in "hex" form
        #filtered_item[0] = "E5"


        print ' '.join(filtered_item)
        sectorcount = sectorcount +1
        #to display 1 sector use the value 32. adjust accordingly"
        if sectorcount ==32:
            break

The result shown were

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 77 8a 1c 22 00 00 00 21
03 00 83 37 ee fb 00 08 00 00 00 b8 3d 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa

As you can see I would need help in editing one of those values in the results (eg changing the value of "77" to maybe "E1")

I tried opening the file as with open('a2.dump', 'wb') as f: but my dump file got nulled. I believe i need to use the write operation to the file but unsure how to do it in Hex aka binary form in Python.

Appreciate any help in advance ! Thanks !

EDIT: As per James Sebastian request the I created a .dump file and edited them in HexEdit with my results shown above.

I then execute the code print repr(open('input.dump', 'rb').read()) Results as shown are:

'\x00w\x8a\x1c"\x00'

The corresponding expected output (the result after the replacements):

'\x00\xe1\x8a\x1c"\x00'

Answer 1

Here's a short demo of doing hex search & replace in a binary file. I took a 32 byte excerpt of your data; here's its hex dump (produced using hd on Linux).

00000000  00 00 00 00 00 00 00 00  77 8a 1c 22 00 00 00 21  |........w.."...!|
00000010  03 00 83 37 ee fb 00 08  00 00 00 b8 3d 00 00 00  |...7........=...|
00000020

Here's the code:

fname = 'qdata'
with open(fname, 'r+b') as f:
    #save position of the start of the data block
    fprev = f.tell()
    stringdata = f.read(32)
    print stringdata.encode('hex')

    #replace the first occurence of \x77\x8a with \xe1\x8a
    newdata = stringdata.replace('\x77\x8a', '\xe1\x8a')
    print newdata.encode('hex')

    #rewind file to the start of the data block
    f.seek(fprev)
    f.write(newdata)

Note that file mode is 'r+b' . This lets us read the file and also modify it. If you open it with a w mode the file is truncated, ie, its previous contents get wiped out, and the file size is reset to zero. If you open it in an a mode the file pointer is positioned at the end of the file to allow data to be appended.

Here's the output that the above code prints:

0000000000000000778a1c220000002103008337eefb0008000000b83d000000
0000000000000000e18a1c220000002103008337eefb0008000000b83d000000

We don't need to do those .encode('hex') and print steps, they're purely informational, so we can see what the program's doing.

Here's the hexdump of the modified file:

00000000  00 00 00 00 00 00 00 00  e1 8a 1c 22 00 00 00 21  |..........."...!|
00000010  03 00 83 37 ee fb 00 08  00 00 00 b8 3d 00 00 00  |...7........=...|
00000020

In the above code I read the entire file contents into RAM; that's certainly not necessary, you can scan it block by block, or however you see fit. But you must do a file .seek() call in between file .read() and .write() operations.

Also, be very careful that you get the positioning correct. And don't accidentally write the wrong data length. It won't change the file length, but it can still make a mess of your file if your replacement data isn't the length you think it is.

---

Here's a function that modifies file data at a given offset. Because its action is potentially dangerous the function prompts the user to make sure that the correct data is being overwritten. In the test code I use the same 32 byte file as before, overwriting the 3 bytes '\\x83\\x37\\xee' at offset 0x12.

def binedit(fname, offset, newdata):
    with open(fname, 'r+b') as f:
        #Show current contents
        f.seek(offset)
        stringdata = f.read(len(newdata))
        print 'Current data:'
        print '%08X: %s\n' % (offset, stringdata.encode('hex'))

        prompt = 'Replace with %s ? (y/N) ' % newdata.encode('hex')
        s = raw_input(prompt)
        if s != 'y':
            print 'Aborting'
            return

        #Replace data at offset with newdata
        f.seek(offset)
        f.write(newdata)


fname = 'qdata'
offset = 0x12
newdata = 'dead42'.decode('hex')
binedit(fname, offset, newdata)

output

Current data:
00000012: 8337ee

Replace with dead42 ? (y/N) y

The "before" and "after" hex dumps:

00000000  00 00 00 00 00 00 00 00  77 8a 1c 22 00 00 00 21  |........w.."...!|
00000010  03 00 83 37 ee fb 00 08  00 00 00 b8 3d 00 00 00  |...7........=...|
00000020

00000000  00 00 00 00 00 00 00 00  77 8a 1c 22 00 00 00 21  |........w.."...!|
00000010  03 00 de ad 42 fb 00 08  00 00 00 b8 3d 00 00 00  |....B.......=...|
00000020

Disclaimer: If you destroy valuable data using this code it's not my fault!

Answer 2

To replace a byte in a binary file, you don't need a hex dump eg, to replace b'\\x77' with b'\\xE1' :

#!/usr/bin/env python
import mmap
from contextlib import closing

with open('a2.dump', 'r+b') as file, \
     closing(mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_WRITE)) as s:
    i = -1
    while 1:
        i = s.find(b'\x77', i+1)
        if i < 0: # not found
            break
        s[i] = b'\xE1'[0] # replace

It performs the replacements inplace. It works for arbitrary large files.

For example, if the input file is created using:

open('a2.dump','wb').write(b'\x00w\x8a\x1c"\x00')

then the output (after the 77 -> E1 replacement) is:

print(repr(open('a2.dump','rb').read()))
# -> b'\x00\xe1\x8a\x1c"\x00'

Notice that 0x77 byte is replaced with 0xE1 .

See Python - How can I change bytes in a file .