Scanner stops reading randomly when ran from exported .jar but not when ran from Eclipse

Question

so today I ran into some trouble while using the java Scanner. I use the Scanner class many times in my project and I never ran into any problem.

Basically, I always do something like this:

try (Scanner scanner = new Scanner(file)) {
    while(scanner.hasNextLine()) {
        String line = scanner.nextLine();
        ...
    }
} catch ...
} finally ...

and the Scanner works just fine because it's just some simple code. Today, however, I used the code above to read text files with about 17000 lines.

At first the code worked just fine (when running it through Eclipse) as I expected but then, after exporting the project, the Scanner would stop reading after about 400 lines.

I googled a bit and in the end I solved the problem thanks to these answers:

• Answer #1
• Answer #2

All I had to do was change the constructor from

Scanner scanner = new Scanner(file)

to

Scanner scanner = new Scanner(new FileInputStream(sql)))

It is some weird encoding problem, I get it. But why when I ran the code from Eclipse it worked flawlessy and when I ran it from my exported jar the Scanner stopped after reading about 400 lines?

The code does the exact same thing in both cases because I set up Eclipse so that it would use the same working directory as the exported .jar archive (because it has got some data subdirectories):

1. Takes the same .gz archive
2. Extract a file from the .gz
3. Read the file like I showed above

Not sure if it helps but Eclipse is set up to save source files in UTF-8 format.

Thanks in advance

Answer 1

But why when I ran the code from Eclipse it worked flawlessy and when I ran it from my exported jar the Scanner stopped after reading about 400 lines?

Scanner has two different constructors that accept a File as argument. From the docs :

Scanner(File source)

Bytes from the file are converted into characters using the underlying platform's default charset .

and

Scanner(File source, String charsetName)

Bytes from the file are converted into characters using the specified charset .

So if you do not specify the charsetName , it will use the environment's default charset.

The environment encoding when you run your project outside Eclipse is probably other than UTF-8. To check that this is the case you can write a simple program like this:

class CheckDefaultCharset {
    public static void main(String... args) {
        System.out.println(Charset.defaultCharset());
    }
}

And run it on both environments.

For example, when running the above code from Eclipse I get:

UTF-8

And when running in the PowerShell (Windows 7), I get:

windows-1252

To avoid this type of problem it would be better to always specify the encoding of the files you intend to use when using Scanner .