I have an Item
object as below:
class Item{
private String idProperty;
private String prop1Db1;
private String prop2Db1;
private String prop1Db2;
private String prop2Db2;
// getters and setters here
}
Now, for a given idProperty
, two of the properties comes from one database, while other two properties comes from another database (as clear from the property names). The tables in both the databases have idProperty
as the common property.
One approach I could take is read the first three properties from the first database. Pass the incomplete Item
to a Processor
and fire a query in the Processor
to get other two properties to complete the Item
before passing it to Writer
.
However, for a huge number of records (millions), it will be a huge performance hit just with the number of queries I will have to fire in the Processor
.
Is there any way to do it in the Reader
itself ? A Reader
which reads from two database simultaneously to create an Item
before sending it to Processor
and Writer
ahead ?
Take this answer as a 'workaround' because I think more elegant solutions are around the corner...
You should change your reader/processor/writer let them return a List<Item>
instead of a single Item
object; in this way in your processor you can fire a single query using the SQL IN()
fetching Item.idProperty
from processing List<>
.
According to List<>
's size you can reduce the amount of fired queries.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.