使用Java + SAMBA + Win 7的文件列表非常慢

Question

I have identified that calling Java's File.listFiles() over a SAMBA folder from a Win 7 client is extremely slow , actually almost 100 times slower than from a Mac client. (Both clients are running Java 7). Both clients are naturally on the same 1GB LAN. A customer has tested from his Win XP client and received far better performance than from his Win7 client. Does anyone know how to resolve this?

Some test data, including timing: Listing 31 folders over SAMBA ("QNAP" NAS disk):

Win 7:

listing \\nas\Public\David\Remote Stocks: 0.0s
listing \\nas\Public\David\Remote Stocks\Animals: 0.093s
listing \\nas\Public\David\Remote Stocks\Animals\.jalbum: 0.312s
listing \\nas\Public\David\Remote Stocks\Animals\.jalbum\thumbs: 0.405s
listing \\nas\Public\David\Remote Stocks\album: 1.248s
listing \\nas\Public\David\Remote Stocks\album\Animals: 1.31s
listing \\nas\Public\David\Remote Stocks\album\Animals\thumbs: 1.388s
listing \\nas\Public\David\Remote Stocks\album\Animals\slides: 1.762s
listing \\nas\Public\David\Remote Stocks\album\Animals\res: 2.542s
listing \\nas\Public\David\Remote Stocks\album\Sports: 2.761s
listing \\nas\Public\David\Remote Stocks\album\Sports\thumbs: 2.839s
listing \\nas\Public\David\Remote Stocks\album\Sports\slides: 3.229s
listing \\nas\Public\David\Remote Stocks\album\Sports\res: 3.978s
listing \\nas\Public\David\Remote Stocks\album\res: 4.196s
listing \\nas\Public\David\Remote Stocks\album\Scenic: 4.695s
listing \\nas\Public\David\Remote Stocks\album\Scenic\thumbs: 4.773s
listing \\nas\Public\David\Remote Stocks\album\Scenic\slides: 5.194s
listing \\nas\Public\David\Remote Stocks\album\Scenic\res: 5.99s
listing \\nas\Public\David\Remote Stocks\album\People: 6.208s
listing \\nas\Public\David\Remote Stocks\album\People\thumbs: 6.302s
listing \\nas\Public\David\Remote Stocks\album\People\slides: 6.692s
listing \\nas\Public\David\Remote Stocks\album\People\res: 7.472s
listing \\nas\Public\David\Remote Stocks\.jalbum: 7.659s
listing \\nas\Public\David\Remote Stocks\Sports: 7.768s
listing \\nas\Public\David\Remote Stocks\Sports\.jalbum: 7.909s
listing \\nas\Public\David\Remote Stocks\Sports\.jalbum\thumbs: 7.987s
listing \\nas\Public\David\Remote Stocks\Scenic: 8.486s
listing \\nas\Public\David\Remote Stocks\Scenic\.jalbum: 8.642s
listing \\nas\Public\David\Remote Stocks\Scenic\.jalbum\thumbs: 8.72s
listing \\nas\Public\David\Remote Stocks\People: 9.282s
listing \\nas\Public\David\Remote Stocks\People\.jalbum: 9.422s

Same listing on Mac:

listing /Volumes/Public/David/Remote Stocks: 0.0s
listing /Volumes/Public/David/Remote Stocks/Animals: 0.013s
listing /Volumes/Public/David/Remote Stocks/Animals/.jalbum: 0.018s
listing /Volumes/Public/David/Remote Stocks/Animals/.jalbum/thumbs: 0.022s
listing /Volumes/Public/David/Remote Stocks/album: 0.027s
listing /Volumes/Public/David/Remote Stocks/album/Animals: 0.03s
listing /Volumes/Public/David/Remote Stocks/album/Animals/thumbs: 0.032s
listing /Volumes/Public/David/Remote Stocks/album/Animals/slides: 0.034s
listing /Volumes/Public/David/Remote Stocks/album/Animals/res: 0.038s
listing /Volumes/Public/David/Remote Stocks/album/Sports: 0.04s
listing /Volumes/Public/David/Remote Stocks/album/Sports/thumbs: 0.042s
listing /Volumes/Public/David/Remote Stocks/album/Sports/slides: 0.046s
listing /Volumes/Public/David/Remote Stocks/album/Sports/res: 0.05s
listing /Volumes/Public/David/Remote Stocks/album/res: 0.052s
listing /Volumes/Public/David/Remote Stocks/album/Scenic: 0.058s
listing /Volumes/Public/David/Remote Stocks/album/Scenic/thumbs: 0.064s
listing /Volumes/Public/David/Remote Stocks/album/Scenic/slides: 0.068s
listing /Volumes/Public/David/Remote Stocks/album/Scenic/res: 0.074s
listing /Volumes/Public/David/Remote Stocks/album/People: 0.08s
listing /Volumes/Public/David/Remote Stocks/album/People/thumbs: 0.082s
listing /Volumes/Public/David/Remote Stocks/album/People/slides: 0.085s
listing /Volumes/Public/David/Remote Stocks/album/People/res: 0.089s
listing /Volumes/Public/David/Remote Stocks/.jalbum: 0.091s
listing /Volumes/Public/David/Remote Stocks/Sports: 0.103s
listing /Volumes/Public/David/Remote Stocks/Sports/.jalbum: 0.106s
listing /Volumes/Public/David/Remote Stocks/Sports/.jalbum/thumbs: 0.108s
listing /Volumes/Public/David/Remote Stocks/Scenic: 0.11s
listing /Volumes/Public/David/Remote Stocks/Scenic/.jalbum: 0.122s
listing /Volumes/Public/David/Remote Stocks/Scenic/.jalbum/thumbs: 0.124s
listing /Volumes/Public/David/Remote Stocks/People: 0.126s
listing /Volumes/Public/David/Remote Stocks/People/.jalbum: 0.133s

I finally attempted to perform a multi threaded listing on Windows to overcome network lag effects. It helped somewhat, -from 9 to 3 seconds, ie 3 times faster, but still significantly slower performance than the 0.133s measured from my MacBook pro client.

Answer 1

I think you stumbled upon a known Windows issue.

Maybe this article could help you: http://www.sysprobs.com/windows-7-network-slow

EDIT:

If you are sure that your issue is not due to a bad behaviour of the operating system when it connects to shares exposed by older SMB protocols, then it's a (not unheard of) slowness imputable to the JDK implementation.

OpenJDK's and Oracle Java's java.io.File.listFiles() delegates to the String [] list() method of the (unexposed) abstract class java.io.FileSystem which in turn is implemented in platform specific manner by java.io.UnixFileSystem , java.io.Win32FileSystem , and java.io.WinNTFileSystem .

Their list() function is always native , so I think the slowness is imputable to the DLL implementation that ships with the JRE.

We had before similar issues regarding file browsing in Windows XP. It was an outstanding bug of the JRE for much time.

Answer 2

Problem solved by using the Files.walkFileTree API. With it I'm able to grab all attributes of all files of a folder with one SAMBA network IO call.