简体   繁体   English

保持通过JDBC插入大量数据的最佳方法。 批量插入还是存储过程?

[英]Best way to keep inserting large volume of data via JDBC. Batch insert or stored procedure?

I'm using Spring connecting to Sql Server 2008 R2 via JDBC . 我正在使用Spring通过JDBC连接到Sql Server 2008 R2

All I need is to insert a large amount of data to a table in the database as fast as possible. 我需要的是尽可能快地将大量数据插入数据库中的表。 I'm wondering which way is better: 我想知道哪种方法更好:

  1. Use Spring batch insert mention here 使用Spring批处理插件在这里提到

  2. Create stored procedure in database and call it on Java side 在数据库中创建存储过程并在Java端调用它

Which one is better? 哪一个更好?

It depends on two things stored producer would take up the database time where as batch would take up time on the program side. 这取决于存储的生产者将占用数据库时间的两件事,而批处理则将在程序侧占用时间。 so depending on what you are more concerned with it is really up to up. 因此,取决于您更关心的是什么。 Me i would prefer the batch as to keep the database time free reducing errors that might occur. 我希望批处理可以节省数据库时间,以减少可能发生的错误。 Hope this helps! 希望这可以帮助!

Spring batch is an excellent framework and it can be used as an ETL (Extract, Transform, Load) tool with respect to database. Spring批处理是一个出色的框架,可以用作数据库的ETL(提取,转换,加载)工具。

Spring batch divides any import job in 3 steps: 1. Read : Read data from any source. Spring批处理将任何导入作业分为3个步骤:1.读取:从任何来源读取数据。 It can be any other database, any file (XML, CSV or any other) or anything else 2. Process: Process input data, validate it and may convert it to your required objects. 它可以是任何其他数据库,任何文件(XML,CSV或任何其他文件)或其他任何文件。2.处理:处理输入数据,对其进行验证并将其转换为所需的对象。 3. Save: Save data into database or any custom file format 3.保存:将数据保存到数据库或任何自定义文件格式

Spring batch is useful when you need long running jobs with restart/resume capabilities. 当您需要具有重新启动/恢复功能的长时间运行的作业时,春季批处理很有用。 Also it is lot slower that any direct DB import tool like impdp for Oracle. 而且,任何直接的数据库导入工具(如impdp for Oracle)也要慢得多。 Spring batch saves its state in database so it is an overhead and consumes long time. Spring批处理将其状态保存在数据库中,因此这是一项开销,并且会花费很长时间。 However you can hack spring batch and make it not save the state in DB but it costs loss of restart/resume capabilities. 但是,您可以修改spring batch,并使其不将状态保存到DB中,但是会损失重新启动/恢复功能的成本。

So if speed is your prime requirement, you should choose some database specific option. 因此,如果速度是您的主要要求,则应选择一些特定于数据库的选项。 But if you need do some validation and/or processing Spring batch is an excellent option, you just need to configure it properly. 但是,如果您需要进行一些验证和/或处理,那么Spring batch是一个很好的选择,您只需要正确配置它即可。 Also Spring batch provides scalability and database independence. Spring批处理还提供了可伸缩性和数据库独立性。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 通过JDBC在MSSQL中运行存储过程的正确方法 - Correct Way to Run Stored Procedure In MSSQL Via JDBC 通过JDBC通过MYSQL存储过程插入朝鲜语字符会出错 - Inserting Korean characters via JDBC through MYSQL stored procedure giving error 使用Spring JDBC API批量执行存储过程 - Batch executing of stored procedure with Spring JDBC API JDBC:批量插入未将值插入数据库 - JDBC : Batch insert not inserting value to database Spring JDBC。 插入具有一对多关系的记录 - Spring JDBC. Insert record with one to many relationship Sybase 存储过程插入限制与 JDBC 驱动程序 - Sybase stored procedure insert limit with JDBC driver Spring Batch - 验证数据加载/批量插入的最佳方法? - Spring Batch - best way to validate data load/batch insert? 通过JDBC将长度可变的外键列表作为参数传递给存储过程,然后使用列表将行插入表中 - Pass in variable length lists of foreign keys as arguments to a stored procedure via JDBC and then use the lists to insert rows into a table spring 数据 jdbc。 无法为枚举添加自定义转换器 - spring data jdbc. Can't add custom converter for enum 在Oracle JDBC中是否可以批量存储过程调用并检索OUT参数? - in Oracle JDBC is it possible to batch stored procedure calls and retrieve OUT parameters?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM