[英]Remove duplicates from PreparedStatement batch JDBC
I have a batch which I need to use in order to update a table. 我有一个批处理,需要使用它来更新表。 This batch is getting fill by an algorithm that I can't change. 我无法更改的算法填补了这一批次。 But it is doing something like this: 但是它正在做这样的事情:
String updateFilter = "UPDATE payload_roas SET filtered = TRUE WHERE asn = ? AND prefix= ? AND max_length = ?";
PreparedStatement ps= connection.prepareStatement(updateFilter);
for(int i = 0; i < roas.size(); i++) {
roa = roas.get(i);
ps.setLong(1, roa.getAsn());
ps.setObject(2, roa.getPrefix(), OTHER);
ps.setInt(3, roa.getMax_length());
ps.setBoolean(4, roa.isWhitelist);
ps.setBoolean(5, roa.isFilter);
ps.addBatch();
}
This batch is quite big ~50K entries.So no surprise that it takes a lot of time when I'm doing executeBatch. 这个批处理大约有5万个条目,所以当我执行executeBatch时会花费很多时间也就不足为奇了。 However this ps contains lots of duplicates which means it is doing multiple redundant updates. 但是,此ps包含大量重复项,这意味着它正在执行多个冗余更新。 Is there any way to perform a distinct operation on this batch in order to remove this duplicates? 有什么方法可以对此批处理执行不同的操作以删除重复项吗?
You can try to do something like this: 您可以尝试执行以下操作:
String updateFilter = "UPDATE payload_roas SET filtered = TRUE WHERE asn = ? AND prefix= ? AND max_length = ?";
PreparedStatement ps= connection.prepareStatement(updateFilter);
HashSet<String> hashKeys = new HashSet<>();
for(int i = 0; i < roas.size(); i++) {
roa = roas.get(i);
String key = roa.getAsn() + roa.getPrefix().toString() + roa.getMax_length() + roa.isWhitelist + roa.isFilter;
if (!hashKeys.contains(key)) {
hashKeys.add(key);
ps.setLong(1, roa.getAsn());
ps.setObject(2, roa.getPrefix(), OTHER);
ps.setInt(3, roa.getMax_length());
ps.setBoolean(4, roa.isWhitelist);
ps.setBoolean(5, roa.isFilter);
ps.addBatch();
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.