strtok function of C definition

Question

The following is an example of the function strok from http://www.cplusplus.com/reference/cstring/strtok/?kw=strtok

Does anyone know why

pch = strtok (NULL, " ,.-");

is used instead of

pch = NULL;

?

/* strtok example */
#include <stdio.h>
#include <string.h>

int main ()
{
  char str[] ="- This, a sample string.";
  char * pch;
  printf ("Splitting string \"%s\" into tokens:\n",str);
  pch = strtok (str," ,.-");
  while (pch != NULL)
  {
    printf ("%s\n",pch);
    pch = strtok (NULL, " ,.-");
  }
  return 0;
}

Answer 1

When you pass NULL as the first argument to strtok() , it's an indication that you want to carry on with the current tokenising operation (ie, get the next matching token). It maintains certain state information between calls which makes this possible.

It looks like you may be thinking that:

pch = strtok (NULL, " ,.-");

will tokenise the NULL string but that's not actually how it works. The strtok() function uses that argument to decide whether you're starting a new tokenising operation (non- NULL ) or continuing with the current one ( NULL ).

That's totally different to setting the pointer to NULL . Doing so would only get the first token from that string rather than all of them.

Answer 2

strtok() maintains an internal reference to the last thing it parsed. Calling strtok(NULL, " ,.-") keeps parsing the unparsed remainder of the string. Checking if the return value is NULL (as stored in pch ) tells you when there is no more string left to parse. Setting pch to NULL explicitly would end the loop after a single iteration since parsing the first n-1 tokens will return a non- NULL value.

Answer 3

The first call in strtok() must have a string, which it will parse to get the first token string. The strok() will maintain the remainder of the unparsed string internally. So, setting NULL in the first argument of strtok() is used to indicate that carry on with the current string. In case a new string is provided, strtok() will remove the unparsed string and will go forward with the new string.

Setting NULL to pch and then calling strtok() will not change the behavior.

Answer 4

In your reference link:

On a first call , the function expects a C string as argument for str , whose first character is used as the starting location to scan for tokens. In subsequent calls , the function expects a null pointer and uses the position right after the end of the last token as the new starting location for scanning.

Also note that pch is the token itself returned by the strtok .

Note: Read the answer paying attention to the highlights.