[英]How do you update an existing avro schema using apache avro SchemaBuilder?
I am testing a new schema registry which loads and retrieves different kinds of avro schemas. 我正在测试一个新的架构注册表,该架构可以加载和检索不同类型的Avro架构。 In the process of testing, I need to create a bunch of different types of avro schemas. 在测试过程中,我需要创建一堆不同类型的Avro模式。 As it involves a lot of permutations, I decided to create the schema programmatically. 由于涉及许多排列,因此我决定以编程方式创建模式。 I am using the apache avro SchemaBuilder to do so. 我正在使用apache avro SchemaBuilder来这样做。
I created the avro using : 我使用创建了avro:
Schema oldSchema = SchemaBuilder
.record("abc")
.aliases("records")
.fields()
.name("field_null")
.type("null")
.noDefault()
.endRecord();
This worked. 这工作了。 The avro created looks like : 创建的avro如下所示:
{
"type" : "record",
"name" : "abc",
"fields" : [ {
"name" : "field_null",
"type" : "null"
} ],
"aliases" : [ "records" ]
}
Now I want to create a new version of the schema using the apache avro libraries like : 现在,我想使用apache avro库来创建架构的新版本:
{
"type" : "record",
"name" : "abc",
"fields" : [ {
"name" : "field_null",
"type" : "null"
},
{
"name" : "new_field",
"type" : "int",
"default" : 10
}
],
"aliases" : [ "records" ]
}
For this, I tried : 为此,我尝试了:
Schema.Field field = new Schema.Field("new_field", SchemaBuilder.builder().intType(),
"NewField", 10);
List<Schema.Field> fields = new ArrayList<>();
fields.add(field);
fields.addAll(oldSchema.getFields());
Schema record = Schema.createRecord(oldSchema.getName(),
"Changes",
oldSchema.getNamespace(),
false,
fields);
I get : 我得到:
org.apache.avro.AvroRuntimeException: Field already used: field_null type:NULL pos:0
at org.apache.avro.Schema$RecordSchema.setFields(Schema.java:647)
at org.apache.avro.Schema$RecordSchema.<init>(Schema.java:618)
at org.apache.avro.Schema.createRecord(Schema.java:167)
My problem is : 我的问题是:
You can try this to create fields, maybe it's clumsy : 您可以尝试创建字段,这可能很笨拙:
Schema.Field field = new Schema.Field("new_field",SchemaBuilder.builder().intType(),
"NewField", 10);
List<Schema.Field> fields = new ArrayList<>();
for (Schema.Field f : oldSchema.getFields()) {
Schema.Field _field = new Schema.Field(f.name(), f.schema(), f.doc(), f.defaultValue());
fields.add(_field);
}
To copy the fields from the old schema to the new one you have to do a deep copy of each field as @xiping xing suggested. 要将字段从旧模式复制到新模式,您必须按照@xiping xing的建议对每个字段进行深度复制。
This is because the Schema
class checks that the field is only added once to a schema, and in your case those fields were already added to the old schema. 这是因为Schema
类检查该字段仅添加到架构一次,并且在您的情况下,这些字段已经添加到旧架构。
You can see how they use a flag in this snippet from Avro 1.7.7
: 您可以在Avro 1.7.7
此代码段中看到他们如何使用标志:
@Override
public void setFields(List<Field> fields) {
if (this.fields != null) {
throw new AvroRuntimeException("Fields are already set");
}
int i = 0;
fieldMap = new HashMap<String, Field>();
LockableArrayList ff = new LockableArrayList();
for (Field f : fields) {
if (f.position != -1)
throw new AvroRuntimeException("Field already used: " + f);
f.position = i++;
final Field existingField = fieldMap.put(f.name(), f);
if (existingField != null) {
throw new AvroRuntimeException(String.format(
"Duplicate field %s in record %s: %s and %s.",
f.name(), name, f, existingField));
}
ff.add(f);
}
this.fields = ff.lock();
this.hashCode = NO_HASHCODE;
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.