Splitting a line from a file into tokens in java

Question

I am currently reading the file token by token and I want to read a whole line first and then split it up into tokens instead. This is part of the code where I am reading by token:

public void getNextToken(){
     String temp = input.next();        

     if(temp.equals("$")){
         nextToken = Symbols.ENDOFFILE;
     }
     else if(temp.equals("a")){
         nextToken = Symbols.IDENT;
     }
     else if(temp.equals(";")){
         nextToken = Symbols.SEMICOLON;
     }
     else if(temp.equals("const")){
         nextToken = Symbols.CONSTANT;

Answer 1

Scanner scanner = new Scanner(fileName);
while (scanner.hasNextLine()) {
    String[] tokens = scanner.nextLine().split("\\s");
    //do what you want to do with the tokens
}
scanner.close();

Answer 2

See StringTokenizer :

The Class ctor I am using:

StringTokenizer(String str, String delim)

An example:

 StringTokenizer st = new StringTokenizer("this is a test", "$a; ");
 while (st.hasMoreTokens()) {
     System.out.println(st.nextToken());
 }

Here you can find bunch of examples.

Answer 3

try so:

BuffereReader reader = new BufferedReader(**YourInputStream**);
String line = reader.readLine();

String[] tokens = line.split("\\s");