Error in strtok split using c++

Question

I have to split a sample string using strok function using C++. The sample string is: "This|is||a||sample||string|" , while split this using strok normally.

#include <stdio.h>
#include <string>
#include <string.h>
using namespace std;

int main()
{   
string  str="This||a||sample||string|";

string a;

str=strtok ((char *)str.c_str(),"|");

while (str.c_str() != NULL)
{
    printf ("str:%s\n",str.c_str());

    str = strtok (NULL, "|");

}

return 0;
}

Result:

str:This

str:a
str:sample

str:string

While changing the same string into "This| |a| |sample| |string|" gives the expected result:

str:This

str: 

str:a

str: 

str:sample

str: 

str:string

How can I get the expect result without changing the string?

Answer 1

Using std::strtok on std::string will yield undefined behaviour, since std::strtok is destructive (hint: std::string::c_str() returns const char* ).

Instead, use std::string::find and std::string::substr multiple times:

#include <iostream>
#include <string>
#include <iterator>

template <class OutputIt>
OutputIt safe_tokenizer(const std::string & s, char token, OutputIt out){
  std::string::size_type pos = 0, f;  
  while((f = s.find(token, pos)) != std::string::npos){    
    *out++ = s.substr(pos, f - pos);
    pos = f + 1;
  }
  if(pos < s.size())
    *out++ = s.substr(pos);
  return out;
}

int main(){
  const std::string str = "Hello|World|How|Are|You";
  safe_tokenizer(str, '|', std::ostream_iterator<std::string>(std::cout, "\n"));
  return 0;
}

Answer 2

printf ? strtok ? You still coding in C. And C library is not really a good way (most times) to do things in C++.

In C++ we tend to avoid operating with naked pointers, C arrays and C strings, but use STL or Boost facilities.

Check this thread for a complete examples "in real C++"

Edit: here is another thread , even better.

Edit2: And if you look at right side of this page, you can find column "Related", with many useful links on your topic =)

Answer 3

try at strtok function

char * TOKEN;
char * mystrtok( char * string,
    const char * control)
{
    char * str=NULL;
    if(string == NULL)
    {
        str = TOKEN;
        if(*str == 0)
        return NULL;
    }
    else
        str = string;

    string = str;
    for ( ; *str ; str++ )
    {
        if(*str == *control)
        {   
            *str++ = '\0';
            break;
        }

    }

    TOKEN = str;
    return string;

}