How do you determine token type in Java?

Question

I'm trying to use StringTokenizer and populate 2 stacks with the elements inside the String input. I'm trying to use two if loops embedded in a while loop to populate the right type of token into their respective stacks. I am struggling with how to actually set up a conditional to determine the type. Here's what I have so far.

 Stack numbers = new Stack();
 Stack operators = new Stack();
 StringTokenizer token = new StringTokenizer(expr, delimiters, true);

 while(token.hasMoreTokens()){
   if(token.nextElement() == ){

   }
   if(token.nextElement() == ){

   }
 }

Answer 1

First of all, you shouldn't use == with strings. Please use .equals() instead.

The next problem is that the token you compare in the first condition is another than the one in the 2 condition. token.nextToken() gives returns the first token and also deletes it from the Tokenizer.

You could make a temporary variable which stores the first element. You can then use this one to compare it to something.

Stack numbers = new Stack();
Stack operators = new Stack();
StringTokenizer token = new StringTokenizer(expr, delimiters, true);

 while(token.hasMoreTokens()) {
   String tmp = token.nextElement()
   if(tmp.equals(...) ){

   }
   if(tmp.equals(...){

   }
}

Answer 2

The answer

The returned tokens are actually Strings, so you can do the usual stuff with them. I don't know what your exact requirements are but something like this should work:

List<String> operators = Arrays.asList("+", "-", "*", "/");
StringTokenizer token = ...;

while(token.hasMoreTokens()){
    String next = token.nextElement();
    if(operators.contains(next)){
        //operator
    }
    else if(next.matches("\\d+")){
        //number
    }
    else {
        //error?
    }
 }

First of all you should use a use a variable for the next token, nextElement() also deletes the token from the Tokenizer so you have to keep it somewhere. Then we can determine the type by comparing String, here I check if it's part of a list of predetermined operators and whether it matches the regex \\d+ , ie. whether it is a number.

Deprecation

The StringTokenizer JavaDoc says it has been deprecated in favor of String.split :

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

So I would recommend against using this class.