Java scanner won't match square brackets

Question

I am having trouble parsing python like lists with scanners in Java. For some reason I seem to be doing something wrong when attempting to detect the square brackets. What puzzles me most is that it seems that somehow the occurences of square brackets in the input triggers the unwanted behaviour.

Scanner scanner1 = new Scanner("a[");
assert(scanner1.hasNext("a\\[")); // Does not trigger assertion

Scanner scanner2 = new Scanner("[[0]]");
assert(scanner2.hasNext("\\["));  // Triggers assertion, but why?

Scanner scanner3 = new Scanner(" a[[[0]]");
assert(scanner3.hasNext("a\\[")); // Triggers assertion, but why?

This is using

openjdk version "1.8.0_45"
OpenJDK Runtime Environment (build 1.8.0_45-b14)
OpenJDK 64-Bit Server VM (build 25.45-b02, mixed mode)

Answer 1

Scanner

A Scanner does split the input into tokens, splitted by a delimiter (nice explanation http://www.tutorialspoint.com/java/util/java_util_scanner.htm ), by default it is a whitespace. Since you don't have any delimiting characters, the first token is the whole string, so your pattern in hasNext should match the whole string and not only parts of it.

Solution: You need to create correct Regexp Syntax to match the whole string

Scanner scanner1 = new Scanner("a[");
assert(scanner1.hasNext("a\\[")); 

Scanner scanner2 = new Scanner("[[0]]");
assert(scanner2.hasNext("\\[.*"));  

Scanner scanner3 = new Scanner(" a[[[0]]");
assert(scanner3.hasNext("a\\[.*")); 

Another method is to use findInLine

Scanner scanner2 = new Scanner("[[0]]");
String inline = scanner2.findInLine("\\[.*"));  
assert(inline!=null);

Answer 2

The default delimiter of the Scanner is a whitespace, so in the first case the token "a[" is found, but in other cases the tokens are not matched because the expected token is not found: [ is different from [[0]] .

For example, adding the needed whitespaces:

public static void main (String[] args) throws java.lang.Exception  {
    Scanner scanner1 = new Scanner("a[");
    System.out.println(scanner1.hasNext("a\\[")); //true        
    Scanner scanner2 = new Scanner("[ [0]]");
    System.out.println(scanner2.hasNext("\\["));  //true
    Scanner scanner3 = new Scanner(" a[ [ [0]]"); 
    System.out.println(scanner3.hasNext("a\\[")); //true
}

In other words, the expected token must match totally, not partially, and the separator must be properly defined. You can play with this code here