Tokenizing string loop memory errors

Question

I'm looping through an array, trying to take each token and insert into another string array (char**) and I'm getting invalid writes from valgrind as well as use of uninitialized value. How would I fix this?

        char *tstring;
        int i = 0;
        char **tokens = (char **)malloc(sizeof(contents));
        tstring = strtok(contents, "\"(),-> ");
        printf("sizeof(tstring) = %ld\tsizeof(*tstring) = %ld\nsizeof(contents) = %ld\n", sizeof(tstring), sizeof(*tstring), sizeof(contents));
        tokens[i] = (char*)malloc(sizeof(tstring));
        printf("tstring address: %p\ntokens address: %p\ntokens[i] address: %p\n",tstring,tokens, tokens[i]);
        strcpy(tokens[i], tstring);
        printf("token[0]: %s\n", tokens[i]);
        while( tokens[i] != NULL ) {
                i++;
                tstring = strtok(NULL, "\"(),-> ");
                if(tstring != NULL)
                        printf("token[%d]: %s\n", i, tstring);
                tokens[i] = (char*)malloc(sizeof(tstring));
                strcpy(tokens[i], tstring);
        }

Here's the string being tokenized

"a" -> ("boo", 1), ("baa", 1)
"baa" -> ("baa", 1)
"boo" -> ("boo", 1)
"cat" -> ("baa", 1)
"dog" -> ("boo", 1)
"name" -> ("boo", 2), ("baa", 1)

And here's the valgrind output

sizeof(tstring) = 8 sizeof(*tstring) = 1
sizeof(contents) = 8
tstring address: 0x51f1041
tokens address: 0x51f1490
tokens[i] address: 0x51f14e0
token[0]: a
token[1]: boo
==4101== Invalid write of size 8
==4101==    at 0x400F3B: Create_List_Container (search.c:166)
==4101==    by 0x4012D5: main (search.c:234)
==4101==  Address 0x51f1498 is 0 bytes after a block of size 8 alloc'd
==4101==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4101==    by 0x400E1A: Create_List_Container (search.c:154)
==4101==    by 0x4012D5: main (search.c:234)
==4101== 
==4101== Invalid read of size 8
==4101==    at 0x400F4F: Create_List_Container (search.c:167)
==4101==    by 0x4012D5: main (search.c:234)
==4101==  Address 0x51f1498 is 0 bytes after a block of size 8 alloc'd
==4101==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4101==    by 0x400E1A: Create_List_Container (search.c:154)
==4101==    by 0x4012D5: main (search.c:234)
==4101== 
==4101== Invalid read of size 8
==4101==    at 0x400F6A: Create_List_Container (search.c:161)
==4101==    by 0x4012D5: main (search.c:234)
==4101==  Address 0x51f1498 is 0 bytes after a block of size 8 alloc'd
==4101==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4101==    by 0x400E1A: Create_List_Container (search.c:154)
==4101==    by 0x4012D5: main (search.c:234)
==4101== 
token[2]: 1
token[3]: baa
token[4]: 1
token[5]: 

token[6]: baa
token[7]: baa
token[8]: 1
token[9]: 

token[10]: boo
token[11]: boo
token[12]: 1
token[13]: 

token[14]: cat
token[15]: baa
token[16]: 1
token[17]: 

token[18]: dog
token[19]: boo
token[20]: 1
token[21]: 

token[22]: name
token[23]: boo
token[24]: 2
token[25]: baa
token[26]: 1
==4101== Use of uninitialised value of size 8
==4101==    at 0x4EBD146: strtok (strtok.S:172)
==4101==    by 0x400EFA: Create_List_Container (search.c:163)
==4101==    by 0x4012D5: main (search.c:234)
==4101== 
==4101== Conditional jump or move depends on uninitialised value(s)
==4101==    at 0x4EBD149: strtok (strtok.S:173)
==4101==    by 0x400EFA: Create_List_Container (search.c:163)
==4101==    by 0x4012D5: main (search.c:234)
==4101== 
token[27]: 

EDIT: Still getting errors

So I adapted the code given to me by netcoder and still getting invalid writes and reads happening at where tokens[i] is getting malloc'd

Here's the code:

    char **tokens = malloc(sizeof(char*)+1);
    if (tokens == NULL) {
            // handle malloc error
            printf("Unable to allocate memory. Exiting...\n");
            exit(0);
    }

    // ...
    while (1) {
            if(i == 0) tstring = strtok(contents, "\"(),-> ");
            else tstring = strtok(NULL, "\"(),-> ");

            if(tstring == NULL) break;

            printf("tstring: %s\tlen(tstring): %d\n", tstring, strlen(tstring));

            tokens[i] = malloc(strlen(tstring)+1);
            if (tokens[i] == NULL) {
                    // handle malloc error
                    printf("Unable to allocate memory. Exiting...\n");
                    exit(0);
            }
            printf("tokens address: %p\t*tokens address: %p\n", tokens, tokens[i]);

            char** tmp = realloc(tokens, (i+2)*sizeof(char*));
            if (tmp == NULL) { 
                    // handle realloc error
                    printf("Unable to reallocate memory. Exiting...\n");
                    exit(0);
            }       
            tokens = tmp;

            strcpy(tokens[i], tstring);
            printf("tokens[%d]: %s\n", i, tokens[i]);
            i++;
    }

Note I allocated **tokens at the start instead of leaving it NULL like netcoder because that was giving me an issue as well.

Here's valgrind:

tstring: a  len(tstring): 1
tokens address: 0x51f1490   *tokens address: 0x51f14e0
tokens[0]: a
tstring: boo    len(tstring): 3
==4609== Invalid write of size 8
==4609==    at 0x400F3E: Create_List_Container (search.c:185)
==4609==    by 0x401388: main (search.c:270)
==4609==  Address 0x51f1538 is 0 bytes after a block of size 8 alloc'd
==4609==    at 0x4C2B7B2: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4609==    by 0x400FB1: Create_List_Container (search.c:193)
==4609==    by 0x401388: main (search.c:270)
==4609== 
==4609== Invalid read of size 8
==4609==    at 0x400F4E: Create_List_Container (search.c:186)
==4609==    by 0x401388: main (search.c:270)
==4609==  Address 0x51f1538 is 0 bytes after a block of size 8 alloc'd
==4609==    at 0x4C2B7B2: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4609==    by 0x400FB1: Create_List_Container (search.c:193)
==4609==    by 0x401388: main (search.c:270)
==4609== 
==4609== Invalid read of size 8
==4609==    at 0x400F77: Create_List_Container (search.c:191)
==4609==    by 0x401388: main (search.c:270)
==4609==  Address 0x51f1538 is 0 bytes after a block of size 8 alloc'd
==4609==    at 0x4C2B7B2: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4609==    by 0x400FB1: Create_List_Container (search.c:193)
==4609==    by 0x401388: main (search.c:270)
==4609== 
tokens address: 0x51f1530   *tokens address: 0x51f1580
==4609== Use of uninitialised value of size 8
==4609==    at 0x4C2BFFC: strcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4609==    by 0x400FF7: Create_List_Container (search.c:201)
==4609==    by 0x401388: main (search.c:270)
==4609== 
==4609== Invalid write of size 1
==4609==    at 0x4C2BFFC: strcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4609==    by 0x400FF7: Create_List_Container (search.c:201)
==4609==    by 0x401388: main (search.c:270)
==4609==  Address 0x0 is not stack'd, malloc'd or (recently) free'd

Fixed, it should be (i+2) in realloc

Answer 1

For starters you do:

tokens[i] = (char*)malloc(sizeof(tstring));

First, don't cast the return value of malloc . Second, you're probably looking for strlen , not sizeof :

tokens[i] = malloc(strlen(tstring)+1); // +1 for the null terminator

...and you do that mistake at least twice.

Then, there's this:

char **tokens = (char **)malloc(sizeof(contents));

...again, casting the return value of malloc , and also that sizeof(contents) is arbitrary because you have no idea how many elements you're going to store in there. This is a good case for realloc :

char **tokens = NULL;
// ...
while (...) {
    // ...
    tokens[i] = malloc(strlen(tstring)+1);
    if (tokens[i] == NULL) {
        // handle malloc error
    }

    char** tmp = realloc(tokens, (i+1)*sizeof(char*));
    if (tmp == NULL) {
        // handle realloc error
    }
    tokens = tmp;

    strcpy(tokens[i], tstring);
    i++;
}

Also note how I moved i++ at the end of the loop to prevent you from accessing tokens[1] when it should be tokens[0] .

Lastly, always check the return value of malloc and realloc .

Answer 2

When you malloc, you're passing sizeof(ptr) , so it's allocating 8 bytes for a 64-bit pointer. You want to malloc(strlen(ptr)+1) and null-terminate it.

Print sizeof(tstring) to the terminal and that should verify it.