[英]Couchbase - Bulk insert via Java
I am currently working on a Java project using Couchbase as a database.我目前正在使用 Couchbase 作为数据库的 Java 项目。 As part of it, I came to create a massive data insert via a json file and a csv file.
作为其中的一部分,我开始通过 json 文件和 csv 文件创建海量数据插入。
And I was wondering, what is the best way to achieve this one?我想知道,实现这一目标的最佳方法是什么? Is it possible to have a sample code?
是否可以提供示例代码?
You have several libraries to perform the conversion between CSV <-> Java object, here is a benchmark of the different libraries:您有几个库来执行 CSV <-> Java object 之间的转换,这里是不同库的基准:
Library![]() |
Read (rec/sec)![]() |
Write (rec/sec)![]() |
Dependencies![]() |
Size (KiB)![]() |
---|---|---|---|---|
Commons CSV![]() |
1,128,102 ![]() |
3,354,703 ![]() |
no![]() |
50 ![]() |
FastCSV![]() |
4,738,726 ![]() |
5,034,953 ![]() |
no![]() |
31 ![]() |
Jackson CSV ![]() |
3,770,602 ![]() |
3,995,294 ![]() |
yes![]() |
2,040 ![]() |
Java CSV ![]() |
1,922,189 ![]() |
2,732,843 ![]() |
no![]() |
13 ![]() |
Opencsv![]() |
1,085,935 ![]() |
1,808,982 ![]() |
yes![]() |
2,625 ![]() |
Sfm+ASM ![]() |
5,164,967 ![]() |
1,901,154 ![]() |
yes![]() |
1.498 ![]() |
Sfm-ASM ![]() |
4,652,517 ![]() |
1,901,154 ![]() |
yes![]() |
1,498 ![]() |
Super CSV![]() |
1,406,090 ![]() |
1,730,984 ![]() |
no![]() |
96 ![]() |
Univocity![]() |
3,594,900 ![]() |
4,050,255 ![]() |
no![]() |
437 ![]() |
For more information on this benchmark you can visit this site: https://github.com/osiegmar/JavaCsvBenchmarkSuite#results有关此基准测试的更多信息,您可以访问此站点: https://github.com/osiegmar/JavaCsvBenchmarkSuite#results
Iterative reading of some CSV data with a header使用 header 迭代读取一些 CSV 数据
NamedCsvReader.builder().build("header 1,header 2\nfield 1,field 2")
.forEach(row -> row.getField("header 2"));
For more information about FastCSV you can visit this site: https://github.com/osiegmar/FastCSV有关FastCSV的更多信息,您可以访问此站点: https://github.com/osiegmar/FastCSV
You have several libraries to perform the conversion between Json <-> Java object, here is a benchmark of the different libraries:您有几个库来执行 Json <-> Java object 之间的转换,这是不同库的基准:
For more information on this benchmark you can visit this site: https://github.com/ngs-doo/dsl-jsons有关此基准测试的更多信息,您可以访问此站点: https://github.com/ngs-doo/dsl-jsons
Conversion from json string to java object:从 json 字符串到 java object 的转换:
String json = "{\"name\":\"Hassan\",\"age\":23}";
Person person = new ObjectMapper().readValue(json, Person.class);
For more information about jackson-databind you can visit this site: https://github.com/FasterXML/jackson-databind有关jackson-databind的更多信息,您可以访问此站点: https://github.com/FasterXML/jackson-databind
Example of Bulk Insert in Java: Java 中的散装插入示例:
protected void doWork() {
final String key = "javaDevguideExampleBulkInsert";
// Create a JSON document content
final JsonObject content = JsonObject.create().put("item", "A bulk insert test value");
// Describe what we want to do asynchronously using RxJava Observables:
ReactiveCollection reactiveCollection = collection.reactive();
Flux<MutationResult> resultFlux = Flux.range(0, 10)
.map(index -> { return key + "_" + index; })
.flatMap(k -> reactiveCollection.upsert(k, content));
resultFlux.subscribe(System.out::println);
}
This code is from official docs-sdk-java此代码来自官方docs-sdk-java
We've recently published a guide to importing which may also be useful: https://docs.couchbase.com/server/current/guides/import.html#importing-using-an-sdk我们最近发布了一个导入指南,它也可能有用: https://docs.couchbase.com/server/current/guides/import.html#importing-using-an-sdk
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.