Solr 4.0 Delta-import 3 tables join

Question

I am using Solr 4.0 DIH(JDBC connector) on Ubuntu. I am trying to make the following MySQL JOIN query work with Solr on Delta-import :

select c.*,u.*,g.* from user u inner join group g on u.bus_grp_id = g.dt_grp_id inner join customer c on c.id = g.dt_id

Here c, u, g are the aliases of the tables customer , user and group respectively.

Below is the data-config.xml file for full and delta import:

<entity name="cust" pk="id" query="select c.*,u.*,g.* from user u inner join group g on u.bus_grp_id = g.dt_grp_id inner join customer c on c.id = g.dt_id"
                deltaImportQuery="select c.*,u.*,g.* from user u inner join group g on u.bus_grp_id = g.dt_grp_id inner join customer c on c.id = g.dt_id where c.id='${dih.delta.c.id}'"
                deltaQuery="select id from customer where last_modified &gt; '${dih.last_index_time}'">

                <entity name="grp" pk="dt_id"
                    query="select * from group where dt_id='${cust.c.id}'"
                    deltaQuery="select dt_id from group where last_modified > '${dih.last_index_time}'"
                    parentDeltaQuery="select id from customer where id=${grp.dt_id}" >

                <entity name="usr" pk="bus_grp_id"
                query="select * from user"
                deltaQuery="select bus_grp_id from user where last_modified > '${dih.last_index_time}'"
                parentDeltaQuery="select dt_grp_id from group where dt_grp_id=${usr.bus_grp_id}" >

</entity>
</entity>
</entity>

The full-import is ok but the Delta-import is not working(I'm getting no results after delta import). It's been almost one month since I'm trying to make this work but couldn't.

Any help? please!

Answer 1

If the query in your main entity element ie

select c.*,u.*,g.* from user u 
inner join group g on u.bus_grp_id = g.dt_grp_id 
inner join customer c on c.id = g.dt_id

is giving you all the fields you need to index in Solr, I don't think you need the other two sub-entities. Also I guess there is a typo in your deltaImportQuery. You should use dih.delta.id and not dih.delta.c.id .

I guess if you keep just this, it should work fine:

<entity 
   name="cust" 
   pk="id" 
   query="select c.*,u.*,g.* from user u 
            inner join group g on u.bus_grp_id = g.dt_grp_id 
            inner join customer c on c.id = g.dt_id"
   deltaImportQuery="select c.*,u.*,g.* from user u 
            inner join group g on u.bus_grp_id = g.dt_grp_id 
            inner join customer c on c.id = g.dt_id 
              where c.id='${dih.delta.id}'"
   deltaQuery="select id from customer 
                where last_modified &gt; '${dih.last_index_time}'" />

As soon as you do a full import, check the file dataimport.properties. Once your full import completes, make sure you got some documents in customer table with last_modified greater than the time you find in dataimport.properties so the delta has some data to run on. Then try delta import and see if those docs get re-indexed. You can find out how many got indexed with the delta import by looking at http://HOST:PORT/solr/CORE/dataimport