searching for a keyword using whitespace analyzer

Question

shown below is my method for indexing my data:

public void getAvailableItems(String sql) {
    Connection con = null;
    Statement stmt = null;
    ResultSet rs = null;
    IndexWriter writer=null;    
    File file = null;
    try{
        file = new File(LUCENE_INDEX_DIRECTORY);
        analyzer = new WhitespaceAnalyzer(Version.LUCENE_CURRENT);
        writer = new IndexWriter(
            FSDirectory.open(file),
            analyzer,
            true,
            IndexWriter.MaxFieldLength.LIMITED
        );

        Class.forName("com.mysql.jdbc.Driver").newInstance();
        //get connection object
        con = DriverManager.getConnection(
            "jdbc:mysql://"+DB_HOST_NAME+"/evergoldbuilders", DB_USER_NAME, DB_PASSWORD);
        //create statement object
        stmt = con.createStatement();
        //execute query
        rs = stmt.executeQuery(sql);
        //iterate through result set
        while(rs.next()){
            String name = rs.getString("category_name").trim() + " " + rs.getString("sub_category_name").trim() + " "  + rs.getString("classification_name").trim() + " "  + rs.getString("item_name").trim();

            Document document = new Document();
            Field nameField = new Field("item_complete_name", name, Field.Store.YES, Field.Index.ANALYZED);
            document.add(nameField);
            writer.addDocument(document);
        }
        writer.optimize();

    }catch(Exception e){
        e.printStackTrace();
    }
}

and my method to search for keywords:

public void searchItem(String column, String search)  throws Exception{

    ScoreDoc[] hits = null;
    QueryParser parser = null;
    Query q = null;

    int hitsPerPage = 50;
    analyzer = new WhitespaceAnalyzer(Version.LUCENE_CURRENT);
    File files = new File(LUCENE_INDEX_DIRECTORY);
    IndexReader reader = IndexReader.open(FSDirectory.open(files),true);
    IndexSearcher searcher = new IndexSearcher(reader);
    TopScoreDocCollector collector = TopScoreDocCollector.create(hitsPerPage, true);
    parser = new QueryParser(Version.LUCENE_CURRENT, "item_complete_name", analyzer);
    q = parser.parse(search + "*");

    searcher.search(q, collector);
    hits = collector.topDocs().scoreDocs;

    System.out.println("Found " + hits.length + " hits.");
    count = 0;
    for(int i=0;i<hits.length;++i) {
        isFound = true;

      int docId = hits[i].doc;
      Document d = searcher.doc(docId);
      System.out.println(d.getField("item_complete_name").stringValue());
      count++;
    }
    searcher.close();
}

and finally, my sample data indexed:

ALUMINUM  4'/O U.S. ALUMINUM
ALUMINUM  4" CHINA ALUMINUM
ALUMINUM  3'/O U.S. ALUMINUM
ALUMINUM  3"A CHINA ALUMINUM
PAINTS DAVIES 4 LITERS DV 472 HI-HEAT RESISTING ALUMINUM (1200°F)
PAINTS DAVIES 4 LITERS DV 470 SILVER FINISH ALUMINUM

my problem is that whenever i search for "alum*", there is no search result found but "aluminum" does. and no search result found on "aluminum AND china*". can i use lucene wildcards (ie * and ?) to search for indexed data using whitespace analyzer? does whitespace analyzer tokenizes non-letters? i want an analyzer to tokenize my data on spaces. is whitespace analyzer the right one to use? thanks a lot!

Answer 1

Use org.apache.lucene.analysis.standard. StandardAnalyzer instead of WhitespaceAnalyzer can solve the problem.