Why am I not able to write in the allocated memory that I created with calloc?

Question

I'm having a problem with a C code that I'm making and I just can't find the root cause. What I'm trying to do is to read a text file from any directory and put all the bytes in the heap by allocating memory with calloc (to have it in 0s)

The problem is that when the file is over a certain size (>=25 KB, could be less) I'm not able to write anything in the allocated memory but I can read the file!

char* ReadBytesFromFile(string fileName[]) {
FILE* fileStream = (FILE*)fopen(fileName, "r");
const fSz fileSizeInBytes = GetFileSizeInBytes(fileName);
char* bytes;
char byteFromFile = 0;

do
{
    bytes = (char*)calloc((size_t)fileSizeInBytes + 1, sizeof(char));
} while (bytes == NULL);


for(long i = 0; i < fileSizeInBytes; i++)
{
    byteFromFile = (char)getc(fileStream);
    if (byteFromFile == EOF) break;
    bytes[i] = byteFromFile;
}

fclose(fileStream);
free(fileStream);

return bytes;
}

By the way:

typedef const char      string;
typedef signed long int fSz;

Another clue is that I'm writing this without an IDE using GCC 11.2.0 with the following flags (in a Makefile):

_compilerOptions = -O0 \
                    -pedantic \
                    -Wall \
                    -Wextra \
                    -Wdiscarded-qualifiers \
                    -Wwrite-strings \
                    -ggdb3 \

I know the code is not "secure" yet, but I don't know what's causing this error. When I'm debugging I'm seeing each character at a time by printing byteFromFile but when I print bytes nothing is being written.

The file size is the only clue that I have, because the code works for files around 1-5 KB but bigger than that doesn't work.

Any ideas or what am I doing wrong? :(

---

Update 1:

I have update the code as follows:

char* ReadBytesFromFile(string fileName[])
{
    FILE* fileStream = (FILE*)fopen(fileName, "rb");
    const fSz fileSizeInBytes = GetFileSizeInBytes(fileName);
    fSz counter = 0;
    char* bytes;
    int byteFromFile = 0;

    do
    {
        bytes = (char*)calloc(fileSizeInBytes + 1, sizeof(char));
    } while (bytes == NULL);
    
    while( (byteFromFile = fgetc(fileStream)) != EOF )
    {
        bytes[counter++] = (char)byteFromFile;
    }

    fclose(fileStream);
    free(fileStream);

    return bytes;
}

but I'm still having the same issue (I'm using GDB):

---

Update 2:

The file that I'm using for test, can be created with the following code:

FILE* fp = (FILE*)fopen("test.txt", "wb");
    char byte = 0x0U;
    
    for(int i = 0; i < 4095; i++)
    {
        (void)putc(byte, fp);
        byte++;
    }

    fclose(fp);

What I'm expecting to see, is the content of the file in the array bytes . Using the code to generate the file I'm using as an example I should see: "\\000\\001\\002\\003\\004\\005\\006\\007\\008\\009\\010\\011\\ ...\\255\\000\\001\\002..." depending on the debugger, GDB prints that information in octal but I wrote it in dec.

It's complicated to upload the whole code, because is split in several files. But basically it should be something like this:

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "main.h"
#include "fileRW.h"

/* ****************************** Typedefs ****************************** */
// Char/strings
typedef unsigned char       uchar;
typedef const char          string;

// Numerical
typedef unsigned char       uint8;
typedef unsigned short      uint16;
typedef unsigned int        uint32;
typedef unsigned long       uint64;

typedef signed char         BYTE;
typedef signed short        WORD;
typedef signed int          DWORD;
typedef signed int long     QWORD;

typedef unsigned char       uBYTE;
typedef unsigned short      uWORD;
typedef unsigned int        uDWORD;
typedef unsigned int long   uQWORD;

typedef signed long int     fSz;

/* ****************************** Function Prototypes ****************************** */
off_t GetFileSizeInBytes(string fileName[]);
char* ReadBytesFromFile(string fileName[]);
void ConvertByteToHexFromFile(uBYTE* inputChar);

int main(int argc, char* argv[])
{
    if(argc > 1)
    {
        for(int i = 1; i < argc; i++)
        {
            (void)ReadBytesFromFile(argv[i]);
        }
    }
    return 0;
}

off_t GetFileSizeInBytes(const char fileName[])
{
    struct stat st;

    if(stat(fileName, &st) == 0)
    {
        return st.st_size;
    }
    else
    {
        return -1;
    }
}

char* ReadBytesFromFile(string fileName[])
{
    FILE* fileStream = (FILE*)fopen(fileName, "rb");
    const fSz fileSizeInBytes = GetFileSizeInBytes(fileName) + 1;
    fSz counter = 0;
    char* bytes;
    int byteFromFile = 0;

    do
    {
        bytes = (char*)malloc(fileSizeInBytes * sizeof(char));
    } while (bytes == NULL);
    
    while( (byteFromFile = fgetc(fileStream)) != EOF )
    {
        bytes[counter++] = (char)byteFromFile;
    }

    //(void)fread(bytes, fileSizeInBytes, 1, fileStream);

    fclose(fileStream);

    return bytes;
}

And the output from GCC -v is:

gcc main.c fileRW.c -c -std=c11 -O0 -pedantic -Wall -Wextra -Wdiscarded-qualifiers -Wwrite-strings -ggdb3 -v  -D _ENABLE_DEBUG_CODE_=0
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-pc-cygwin
Configured with: /mnt/share/cygpkgs/gcc/gcc.x86_64/src/gcc-11.2.0/configure --srcdir=/mnt/share/cygpkgs/gcc/gcc.x86_64/src/gcc-11.2.0 --prefix=/usr --exec-prefix=/usr --localstatedir=/var --sysconfdir=/etc --docdir=/usr/share/doc/gcc --htmldir=/usr/share/doc/gcc/html -C --build=x86_64-pc-cygwin --host=x86_64-pc-cygwin --target=x86_64-pc-cygwin --without-libiconv-prefix --without-libintl-prefix --libexecdir=/usr/lib --with-gcc-major-version-only --enable-shared --enable-shared-libgcc --enable-static --enable-version-specific-runtime-libs --enable-bootstrap --enable-__cxa_atexit --with-dwarf2 --with-tune=generic --disable-bootstrap --enable-languages=c,c++,fortran,lto,objc,obj-c++,jit --enable-graphite --enable-threads=posix --enable-libatomic --enable-libgomp --enable-libquadmath --enable-libquadmath-support --disable-libssp --enable-libada --disable-symvers --with-gnu-ld --with-gnu-as --with-cloog-include=/usr/include/cloog-isl --without-libiconv-prefix --without-libintl-prefix --with-system-zlib --enable-linker-build-id --with-default-libstdcxx-abi=gcc4-compatible --enable-libstdcxx-filesystem-ts
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.2.0 (GCC)
COLLECT_GCC_OPTIONS='-c' '-std=c11' '-O0' '-Wpedantic' '-Wall' '-Wextra' '-Wdiscarded-qualifiers' '-Wwrite-strings' '-ggdb3' '-v' '-D' '_ENABLE_DEBUG_CODE_=0' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-cygwin/11/cc1.exe -quiet -v -dD -idirafter /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/../include/w32api -idirafter /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/lib/../lib/../../include/w32api -D _ENABLE_DEBUG_CODE_=0 main.c -quiet -dumpbase main.c -dumpbase-ext .c -mtune=generic -march=x86-64 -ggdb3 -O0 -Wpedantic -Wall -Wextra -Wdiscarded-qualifiers -Wwrite-strings -std=c11 -version -o /cygdrive/c/Users/Daniel/AppData/Local/Temp/ccIWuk1n.s
GNU C11 (GCC) version 11.2.0 (x86_64-pc-cygwin)
        compiled by GNU C version 11.2.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.24-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-cygwin/11/include-fixed"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/include"
ignoring duplicate directory "/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/lib/../lib/../../include/w32api"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-cygwin/11/include
 /usr/include
 /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/../include/w32api
End of search list.
GNU C11 (GCC) version 11.2.0 (x86_64-pc-cygwin)
        compiled by GNU C version 11.2.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.24-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: befd73bee7080c4d72157ef5c760fe84
COLLECT_GCC_OPTIONS='-c' '-std=c11' '-O0' '-Wpedantic' '-Wall' '-Wextra' '-Wdiscarded-qualifiers' '-Wwrite-strings' '-ggdb3' '-v' '-D' '_ENABLE_DEBUG_CODE_=0' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/as.exe -v --gdwarf-5 -o main.o /cygdrive/c/Users/Daniel/AppData/Local/Temp/ccIWuk1n.s
GNU assembler version 2.37 (x86_64-pc-cygwin) using BFD version (GNU Binutils) 2.37
COLLECT_GCC_OPTIONS='-c' '-std=c11' '-O0' '-Wpedantic' '-Wall' '-Wextra' '-Wdiscarded-qualifiers' '-Wwrite-strings' '-ggdb3' '-v' '-D' '_ENABLE_DEBUG_CODE_=0' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-cygwin/11/cc1.exe -quiet -v -dD -idirafter /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/../include/w32api -idirafter /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/lib/../lib/../../include/w32api -D _ENABLE_DEBUG_CODE_=0 fileRW.c -quiet -dumpbase fileRW.c -dumpbase-ext .c -mtune=generic -march=x86-64 -ggdb3 -O0 -Wpedantic -Wall -Wextra -Wdiscarded-qualifiers -Wwrite-strings -std=c11 -version -o /cygdrive/c/Users/Daniel/AppData/Local/Temp/ccIWuk1n.s
GNU C11 (GCC) version 11.2.0 (x86_64-pc-cygwin)
        compiled by GNU C version 11.2.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.24-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-cygwin/11/include-fixed"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/include"
ignoring duplicate directory "/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/lib/../lib/../../include/w32api"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-cygwin/11/include
 /usr/include
 /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/../include/w32api
End of search list.
GNU C11 (GCC) version 11.2.0 (x86_64-pc-cygwin)
        compiled by GNU C version 11.2.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.24-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: befd73bee7080c4d72157ef5c760fe84
COLLECT_GCC_OPTIONS='-c' '-std=c11' '-O0' '-Wpedantic' '-Wall' '-Wextra' '-Wdiscarded-qualifiers' '-Wwrite-strings' '-ggdb3' '-v' '-D' '_ENABLE_DEBUG_CODE_=0' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/as.exe -v --gdwarf-5 -o fileRW.o /cygdrive/c/Users/Daniel/AppData/Local/Temp/ccIWuk1n.s
GNU assembler version 2.37 (x86_64-pc-cygwin) using BFD version (GNU Binutils) 2.37
COMPILER_PATH=/usr/lib/gcc/x86_64-pc-cygwin/11/:/usr/lib/gcc/x86_64-pc-cygwin/11/:/usr/lib/gcc/x86_64-pc-cygwin/:/usr/lib/gcc/x86_64-pc-cygwin/11/:/usr/lib/gcc/x86_64-pc-cygwin/:/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/
LIBRARY_PATH=/usr/lib/gcc/x86_64-pc-cygwin/11/:/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/lib/../lib/:/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/lib/:/usr/lib/gcc/x86_64-pc-cygwin/11/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-c' '-std=c11' '-O0' '-Wpedantic' '-Wall' '-Wextra' '-Wdiscarded-qualifiers' '-Wwrite-strings' '-ggdb3' '-v' '-D' '_ENABLE_DEBUG_CODE_=0' '-mtune=generic' '-march=x86-64'
windres icon.rs icon.o
gcc main.o fileRW.o icon.o -o code.exe

So far, the code is just starting to take "form" so it's still quite simple. I smashed all the SW units in a single code (just for this question), but now it's in at least 5 files.

And also, I forgot to mention that I'm using Cygwin. Should I change to MinGW?

---

Update 3:

Hello everyone, I have followed some of your recommendations and I think I have found the root cause.

So what I have found is that when a binary file starts with 0x00, this is causing some issue that prevents me to write in my allocated memory. If for example I open a binary file that starts with 0x1 (or >0x0) the code seems to work kind of OK. I've trying to read an executable file for example and for a short period of time seems to be working OK, but as soon as it finds a 0x0 it crashes and it no longer writes to the memory :/

The workaround that I'm implementing (temporarily) is to avoid writing 0s all together. Something like this:

...
while( (byteFromFile = fgetc(fileStream)) != EOF )
    {
        if(byteFromFile != '\0') bytes[counter++] = (char)byteFromFile;
    }
...

This of course will cause a problem, because I will not be able to write 0s (or read them) in my binary files, and that's not very realistic.

---

Update 4:

Hello everyone, the "root cause" is the GBD. If I follow the instructions from the last answer (I can't @ my friend), the "issue" is no longer an "issue".

Anyway, thanks everyone for your time. How can I close this ticket lol?

Answer 1

Any ideas or what am I doing wrong? :(

257

getc() returns 257 different values in the unsigned char range and EOF . Saving in a char loses information and can cause a valid input byte to be misinterpreted as an EOF signal (when char is signed ) or byteFromFile == EOF never to be true (when char is unsigned ).

// char byteFromFile = 0;
int byteFromFile = 0;
...
  // byteFromFile = (char)getc(fileStream);
  byteFromFile = getc(fileStream);
  if (byteFromFile == EOF) break;

An early exit of the loop explains "code works for files around 1-5 KB but bigger than that doesn't work."

Size

Be sure the file size fileSizeInBytes + 1 does not exceed size_t math.

//add
if (fileSizeInBytes < 0 || fileSizeInBytes >= SIZE_MAX) {
  fprintf(stderr, "File size trouble %lld\n", (long long) fileSizeInBytes);
  exit (-1);
}

Once is enough

Repeating a failed memory allocation is not that useful. Very unlikely to work later if it fails the first time.

// do {
//    bytes = (char*)calloc(fileSizeInBytes + 1, sizeof(char));
// } while (bytes == NULL);

bytes = (char*)calloc(fileSizeInBytes + 1, sizeof(char));
if (bytes == NULL) {
  fprintf(stderr, "Out of memory\n");
  exit (-1);
} 

Minor: Cast not needed, size to referenced object

// bytes = (char*)calloc(fileSizeInBytes + 1, sizeof(char));
bytes = calloc(fileSizeInBytes + 1, sizeof *bytes);

Avoid mixing types

Why code types fileSizeInBytes and i differently?
(Looks OK in OP's case as for now , fsz is long ).

const fSz fileSizeInBytes = GetFileSizeInBytes(fileName);
...
//for(long i = 0; i < fileSizeInBytes; i++)
for(fSz i = 0; i < fileSizeInBytes; i++)

Lacking error check

Check to make sure file open was successful.

FILE* fileStream = (FILE*)fopen(fileName, "rb");
// add
if (fileStream == NULL) {
  fprintf(stderr, "Fail to open <%s>\n", fileName);
  exit (-1);
} 

Missing null character

(When malloc() is used.)
Certainly ReadBytesFromFile() should return a pointer to a string . So far it does not as bytes[] lacks a certain null character .

char* ReadBytesFromFile(string fileName[]) {
  ...
  bytes[counter] = '\0'; // add
  return bytes;
}

Casts

OP code is using casts () in various places. IMO, all of them are unnecessary or potentially hiding issues. They all need removal or re-work.

Perhaps some other issue is also occurring

Answer 2

So what I have found is that when a binary file starts with 0x00, this is causing some issue that prevents me to write in my allocated memory. If for example I open a binary file that starts with 0x1 (or >0x0) the code seems to work kind of OK. I've trying to read an executable file for example and for a short period of time seems to be working OK, but as soon as it finds a 0x0 it crashed and it no longer writes to the memory :/

In fact, your code is writing to memory just fine. However, you are inspecting it with print bytes in GDB. Since bytes has type char * , GDB's default is to print it as a string, which means it only prints characters until a null byte is encountered. If the very first byte is null, then GDB prints it as an empty string and ignores anything after that.

To specify how many bytes to dump, you can use the x command instead. For instance, x/17xb bytes will dump the first 17 bytes in hexadecimal; then you should see the actual bytes from your file. If you want to see them as printable characters instead, you can do x/17cb . Use help x for more information.

(As a separate issue, if some other process should write more bytes to the file after your stat call, then your fgetc loop will read too many characters and overrun your buffer. You should keep track of how many bytes you have actually stored, and either quit or enlarge the buffer when it gets full.)