Splitting a user-entered String into separate tokens by white space in C

Question

I am trying to print a user entered string word by word, or tokenize. I have:

char input [1000]; 

char* token;
scanf("%s", input);

token = strtok (input," ,.");

while (token != NULL){
    printf("%s\n",token);
    token = strtok(NULL, " ,.");
}

When I enter something into the console, say "test test one two three.", only the first word is printed out.

Answer 1

You are only scanning in the first word with scanf . scanf expects a formatted string input, ie in your case:

scanf("%s %s %s %s", string1, string2 ...

This isnt useful to you. You should look into using fgets .

Please be aware that gets has no way of limiting input size. It can be very dangerous for your memory if not used only by you. Use fgets .

Here is a live example.

Answer 2

You are on the right track with strtok . While you can use scanf , there is the significant limitation that you must hardcode the maximum number of strings you plan to convert. In the event you use something like:

scanf("%s %s %s %s", string1, string2 ...

Your conversion to tokens will fail for:

one two three four five

So, unless you are guaranteed a set number of strings before you write your code, scanf will not work. Instead, as with your initial attempt, your choice of strtok will provide the flexibility to handle an unlimited number of words.

Your only problem reading input initially was the choice of the scanf file specifier of "%s" where conversion stops when the first whitespace is encountered. If you simply changed your conversion specifier to "%[^\\n]" your would have been able to read all words in the string up to the '\\n' character. However, a better alternative to scanf is probably fgets in this situation. A quick example would be:

#include <stdio.h>
#include <string.h>

#define MAXC 256

int main (void) {

    char buf[MAXC] = {0};
    char *p = buf;

    printf ("\n enter words: ");
    fgets (buf, MAXC, stdin);

    printf ("\n tokens:\n\n");
    for (p = strtok (buf, " "); p; p = strtok (NULL, " \n"))
        printf ("   %s\n", p);

    putchar ('\n');

    return 0;
}

Example/Output

$ ./bin/strtok_fgets

 enter words: a quick brown fox jumps over the lazy dog.

 tokens:

   a
   quick
   brown
   fox
   jumps
   over
   the
   lazy
   dog.

If you would like to use scanf , then you could replace fgets above with scanf ("%255[^\\n]", buf); and accomplish the same thing.