[英]Optimizing a HashMap with iteration
I have a ResultSet
containing 300K records and I am doing the following to iterate it (and other actions once collected). 我有一个包含30万条记录的
ResultSet
,并且我正在执行以下操作来对其进行迭代(以及收集到的其他操作)。 This process takes around 2 minutes to complete. 此过程大约需要2分钟才能完成。 Is there any way to optimize it?
有什么方法可以优化它吗?
Map<String,Map<String,String>> internalMap = new HashMap<String,Map<String,String>>();
while (resultSet.next()) {
final Integer val1 = resultSet.getInt("val1");
final String val2 = resultSet.getString("val2");
final String val3 = resultSet.getString("val3");
final String val4 = resultSet.getString("val4");
final String type = resultSet.getString("type");
final String id = resultSet.getString("id");
addIntern(internalMap,val2,val1,val3,val4,type,id);
}
And the addIntern
method referenced above 以及
addIntern
方法
private static void addIntern(Map<String,Map<String,String>> internalMap, String val2, Integer val1,
String val3,String val4,String type,String id) {
String key = id+"##"+val4;
if (internalMap.get(key) == null) {
internalMap.put(key, new HashMap<String,String>());
}
internalMap.get(key).put("val3", val3);
internalMap.get(key).put("val2", val2);
if("create".equals(type)){
internalMap.get(key).put("create", val1.toString());
}
if("update".equals(type)){
internalMap.get(key).put("update", val1.toString());
}
if("delete".equals(type)){
internalMap.get(key).put("delete", val1.toString());
}
}
Tune the fetch size from your resultSet
; 调整您的
resultSet
的获取大小;
resultSet.setFetchSize(100);
And you could certainly simplify your add method (each get is an O(1)
call, but they add up), like 而且您当然可以简化您的add方法(每个get是一个
O(1)
调用,但它们相加),例如
private static void addIntern(Map<String, Map<String, String>> internalMap, String val2, Integer val1, String val3,
String val4, String type, String id) {
String key = id + "##" + val4;
Map<String, String> kMap;
if (internalMap.containsKey(key)) {
kMap = internalMap.get(key);
} else {
kMap = new HashMap<>();
internalMap.put(key, kMap);
}
kMap.put("val3", val3);
kMap.put("val2", val2);
if ("create".equals(type) || "update".equals(type) || "delete".equals(type)) {
kMap.put(type, val1.toString());
}
}
You won't be able to optimize much since you can't reduce number of iterations. 您将无法优化太多,因为您无法减少迭代次数。
But one change I can see which can be done 但是我可以看到可以做的一项更改
if("create".equals(type)){
internalMap.get(key).put("create", val1.toString());
}
if("update".equals(type)){
internalMap.get(key).put("update", val1.toString());
}
if("delete".equals(type)){
internalMap.get(key).put("delete", val1.toString());
}
Above can be written as below also 上面也可以写成如下
internalMap.get(key).put(type, val1.toString());
This will remove 300k or more if checks. 如果进行检查,这将删除300k或更多。
This will work in a case when your type
can contain only create/update/delete
values. 当您的
type
只能包含create/update/delete
值时,这将起作用。 If it has more you can put a single if
to check if it is equal to any one of 3. 如果更多,则可以输入一个
if
来检查它是否等于3中的任何一个。
Try this. 尝试这个。 (Java 8)
(Java 8)
private static void addIntern(Map<String,Map<String,String>> internalMap,
String val2, Integer val1,
String val3,String val4,String type,String id) {
String key = id+"##"+val4;
Map<String, String> valueMap = internalMap.computeIfAbsent(key, k -> new HashMap<>());
valueMap.put("val3", val3);
valueMap.put("val2", val2);
if("create".equals(type) || "update".equals(type) || "delete".equals(type))
valueMap.put(type, val1.toString());
}
Setting the map initial capacity may help if you have sufficient heap space available: 如果您有足够的可用堆空间,则设置映射初始容量可能会有所帮助:
Map<String,Map<String,String>> internalMap = new HashMap<String,Map<String,String>>(300000*2);
By using the default initial capacity, internalMap must be rehashed many times since you are adding 300000 items. 通过使用默认的初始容量,由于要添加300000项,所以必须多次重新映射internalMap。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.