[英]Oracle insertion with Java spring and hibernate takes too much time
I have a microservice that gets an array of objects from an API and then connects with a remote oracle BBDD and inserts the information.我有一个微服务,它从 API 获取对象数组,然后与远程 oracle BBDD 连接并插入信息。 In a call, I'd get like 7k-10k registers and it's taking on average to save them 1 hour which is crazy.
在一个电话中,我会得到大约 7k-10k 个寄存器,平均要节省 1 小时,这太疯狂了。 Why could it be?
为什么会这样? Is it a problem from the server or I'm doing something wrong?
是服务器的问题还是我做错了什么?
application.properties file application.properties 文件
spring.jpa.database-platform=org.hibernate.dialect.Oracle10gDialect
spring.datasource.url= jdbc:oracle:thin:url
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
spring.jpa.hibernate.ddl-auto=none
hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.default_schema=schema
spring.jpa.show-sql=true
spring.data.rest.basePath=/api
spring.jpa.hibernate.connection.provider_class=org.hibernate.hikaricp.internal.HikariCPConnectionProvider
spring.datasource.hikari.minimumIdle=5
spring.datasource.hikari.maximumPoolSize=20
spring.datasource.hikari.idleTimeout=600000
spring.datasource.hikari.poolName=springBootJPAHikariCP
spring.datasource.hikari.maxLifetime=2000000
spring.datasource.hikari.connectionTimeout=10000
The model of the table表model
@Entity
@Table(name = "CALL")
public class Call {
@Column(name = "ID")
@Id
private String id;
@Column(name = "QUEUEID")
private String queueId;
@Column(name = "QUEUETYPE")
private String queueType;
@Column(name = "AGENTID")
private String agentId;
@Column(name = "DATASET")
private String dataSet;
@Column(name = "CALLDATE")
private Timestamp callDate;
@Column(name = "CALLDURATION")
private String callDuration;
@Column(name = "RINGTIME")
private String ringTime;
@Column(name = "RESULT")
private String result;
@Column(name = "DATE_LOAD")
private Date fecCarga = new Date();
//GETTERS AND SETTERS
JAVA LOGIC JAVA 逻辑
The logic is very simple逻辑很简单
ObjectMapper objectMapper = new ObjectMapper();
String response = reportingServices.getToken();
Token token = objectMapper.readValue(response, Token.class);
String tokenString = token.setToken(token.getToken());
String startDate = req.getStartDate();
String endDate = req.getEndDate() ;
final String uri = API_URL
RestTemplate restTemplate = new RestTemplate();
String result = restTemplate.getForObject(uri, String.class);
JSONObject json = new JSONObject(result);
JSONArray jsonArray = json.getJSONArray("list");
try{
for (int i = 0, size = jsonArray.length(); i < size; i++)
{
JSONObject objectInArray = jsonArray.getJSONObject(i);
Call call = new Call();
String callid = (String) objectInArray.get("callid");
String qid = (String) objectInArray.get("qid");
String type = (String) objectInArray.get("type");
String agent = (String) objectInArray.get("agent");
String dataset = (String) objectInArray.get("dataset");
String datetime = (String) objectInArray.get("datetime");
String duration = (String) objectInArray.get("duration");
String ringtime = (String) objectInArray.get("ringtime");
String resultTable = (String) objectInArray.get("result");
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss");
Date parsedDate = dateFormat.parse(datetime);
Timestamp timestamp = new java.sql.Timestamp(parsedDate.getTime());
call.setId(callid);
call.setQueueId(qid);
call.setQueueType(type);
call.setAgentId(agent);
call.setDataSet(dataset);
call.setCallDate(timestamp);
call.setCallDuration(duration);
call.setRingTime(ringtime);
call.setResult(resultTable);
call.setFecCarga(call.getFecCarga());
callRepository.save(call);
}
return ResponseHandler.generateResponse(HttpStatus.CREATED, "Info created successfully");
} catch (Exception e) {
return ResponseHandler.generateResponse(HttpStatus.INTERNAL_SERVER_ERROR, "Something went wrong.");
}
Repository存储库
import org.springframework.data.jpa.repository.JpaRepository;
public interface callRepository extends JpaRepository<Call,Long> {
}
Even if you configured spring.jpa.properties.hibernate.jdbc.batch_size
it won't help you because you are saving individually each entry of your list. Even if you configured
spring.jpa.properties.hibernate.jdbc.batch_size
it won't help you because you are saving individually each entry of your list. use saveAll()
method from JpaRepository and save in batches of 100 at least.使用 JpaRepository 中的
saveAll()
方法并至少分批保存 100 个。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.