nutch - how to crawl a specific file type?

Question

Is it possible to define a specific file type that will be crawled?

I'm trying to work around the regex-urlfildtr.txt file, but I only see how I can specify which type NOT to crawl.

Is to possible to define the I want to crawl only, say .doc files?

Answer 1

In $NUTCH_HOME/conf/regex-urlfilter.txt file, delete exiting regex patterns and paste this:

+\.doc$ 
-.

This will allow only .doc files to get crawled and filter out rest urls.