[英]strip header from binary file
我有几个演出的原始二进制文件,并且试图将其分块处理。 在开始处理数据之前,我必须删除它具有的标头。 由于原始的二进制文件格式,.find或检查数据块中的字符串之类的字符串方法均无效。 我想自动剥离标头,但它的长度可能会有所变化,而我当前寻找最后一个换行符的方法不起作用,因为原始二进制数据的数据中有匹配的位。
Data format:
BEGIN_HEADER\r\n
header of various line count\r\n
HEADER_END\r\n raw data starts here
我如何阅读文件
filename="binary_filename"
chunksize=1024
with open(filename, "rb") as f:
chunk = f.read(chunksize)
for index, byte in enumerate(chunk):
if byte == ord('\n'):
print("found one " + str(index))
有没有一种简单的方法可以提取HEADER_END \\ r \\ n行而无需在文件中滑动字节数组? 当前方法:
chunk = f.read(chunksize)
index=0
not_found=True
while not_found:
if chunk[index:index+12] == b'HEADER_END\r\n':
print("found")
not_found=False
index+=1
您可以使用线缓存:
import linecache
currentline = 0
while(linecache.getline("file.bin",currentline)!="HEADER_END\n"):
currentline=currentline+1
#print raw data
currentline = currentline + 1
rawdata = linecache.getline("file.bin",currentline)
currentrawdata = rawdata
while(currentrawdata):
currentrawdata = linecache.getline("file.bin",currentline+1)
rawdata = rawdata + currentrawdata
currentline = currentline + 1
print rawdata
更新
我们可以将问题分成两部分,首先可以删除标题,然后将其读取为大块:
lines= open('test_file.bin').readlines()
currentline = 0
while(lines[currentline] != "HEADER_END\r\n"):
currentline=currentline+1
open('newfile.bin', 'w').writelines(lines[currentline:-1])
将创建一个仅包含原始数据的文件(newfile.bin)。 现在可以将其直接读取成块:
chunksize=1024
with open('newfile.bin', "rb") as f:
chunk = f.read(chunksize)
更新2
也可以不使用中间文件来执行此操作:
#defines the size of the chunks
chunksize=20
filename= 'test_file.bin'
endHeaderTag = "HEADER_END\r\n"
#Identifies at which line there is HEADER_END
lines= open(filename).readlines()
currentline = 0
while(lines[currentline] != endHeaderTag):
currentline=currentline+1
currentline=currentline+1
#Now currentline contains the index of the first line to the raw data
#With the reduce operation we generate a single string from the list of lines
#we are considering only the lines after the currentline
header_stripped = reduce(lambda x,y:x+y,lines[currentline:])
#Lastly we read successive chunks and we store them into the chunk list.
chunks = []
reminder = len(header_stripped)%chunksize
for i in range(1,len(header_stripped)/chunksize + reminder):
chunks.append( header_stripped[(i-1)*chunksize:i*chunksize])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.