Equivalent Of Reading In Raw Byte Files In MATLAB for Python

Question

I have raw byte files that have blocks of 28x28 bytes, which represent an image. In each file, there are 1000 of these blocks, which are just data I want to analyze. So for example 'data' has 1000 28x28 byte blocks, which represent 1000 28x28 pixel images.

Right now, I know how to read this in using MATLAB:

fid=fopen(‘data’,’r’); // open the file 
[t1,N]=fread(fid,[28 28],’uchar’); // read in the first example and store it in a 28x28 size matrix t1
[t2,N]=fread(fid,[28 28],uchar); // read the second example into t2 and so on
//To display the image use imshow(t1) or imagesc(t1)

I was wondering how I could do the same with Python. I am having trouble getting it to work.

Answer 1

I got it working by doing this.

fid = open('data/data' + str(digit), 'rb')
dim = np.fromfile(fid, dtype=np.uint8) # read in entire dataset

# loop through the thousand examples and smartly index them to get 28x28 images
for i in xrange(0, 1000):
    # indices to go through data file
    index = i*28*28        
    nextindex = (i+1)*28*28

    # get the images in 28x28 format & convert all pixels to binary
    newdim = dim[index:nextindex]
    newdim = newdim.reshape(28, 28)

Instead of trying to read in the file byte block by byte block, I just read in the entire file and then index through it 28*28 at a time. Please let me know if I am making a mistake anywhere here.