How to tokenize a string containing null values using strtok_r

Question

I have a string which contains some comma separated values. The value may or may not be NULL. like :

strcpy(result, "Hello,world,,,wow,");

I want null values to be printed accepted too. How can I proceed while using strtok_r which gives me NULL values too.

I tried this :

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {

    char result[512] = { 0 };
    strcpy(result, "Hello,world,,,wow");
    char *token, *endToken;
    int i = 0;
    token = strtok(result, ",");
    while (i < 5) {
        printf("%d\n", ++i);
        printf("%s\n", token);
        token = strtok(NULL, ",");
    }
    return 0;
}

and the output is :

1
Hello
2
world
3
wow
4
Segmentation fault (core dumped)

I know why it is giving Segmentation fault. I want the solution so that output is like:

1
Hello
2
World
3
*
4
*
5
wow

I want * to be printed for the null tokens but null tokens are not even extracted.

Answer 1

From strtok_r man page:

A sequence of two or more contiguous delimiter characters in the parsed string is considered to be a single delimiter.

So it won't work in your case. But you can use code like this one:

#include <stdio.h>
#include <string.h>

int main(void) {
    int i = 0;
    char result[512];
    char *str = result, *ptr;
    strcpy(result, "Hello,world,,,wow");
    while (1) {
        ptr = strchr(str, ',');
        if (ptr != NULL) {
            *ptr = 0;
        }
        printf("%d\n", ++i);
        printf("%s\n", str);
        if (ptr == NULL) {
            break;
        }
        str = ptr + 1;
    }
    return 0;
}

Answer 2

If you don't have strsep() you can roll your own.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {

    char result[512] = "Hello,world,,,wow";
    char *token, *endToken;
    int i = 0;

    token = result;
    do {
        endToken = strchr(token, ',');
        if (endToken)
            *endToken = '\0';           // terminate token
        printf("%d\n", ++i);
        if (*token == '\0')             // substitute the empty string
            printf("*\n");
        else
            printf("%s\n", token);
        if (endToken)
            token = endToken + 1;
    } while (endToken);
    return 0;
}

Program output:

1
Hello
2
world
3
*
4
*
5
wow

Answer 3

For strtok to find a token, there must be a first character that is not a delimiter. It only returns NULL when it reaches the end of the string, ie when it finds the '\\0' character.

To determine the beginning and the end of a token, the function first scans from the starting location for the first character not contained in delimiters (which becomes the beginning of the token) . And then scans starting from this beginning of the token for the first character contained in delimiters, which becomes the end of the token. The scan also stops if the terminating null character is found.

http://www.cplusplus.com/reference/cstring/strtok