[英]Can we make Lucene IndexWriter serializable for ExecutionContext of Spring Batch?
This question is related to my another SO question . 这个问题与我的另一个SO问题有关 。
To keep IndexWriter open for the duration of a partitioned step, I thought to add IndexWriter
in ExecutionContext
of partitioner and then close in a StepExecutionListenerSupport
's afterStep(StepExecution stepExecution)
method. 为了使IndexWriter在分区步骤的整个过程中保持打开状态,我想在分区器的ExecutionContext
中添加IndexWriter
,然后在StepExecutionListenerSupport
的afterStep(StepExecution stepExecution)
方法中afterStep(StepExecution stepExecution)
。
Challenge that I am facing in this approach is that ExecutionContext
needs Objects to be serializable. 我用这种方法面临的挑战是ExecutionContext
需要对象可序列化。
In light of these two questions, Q1 , Q2 -- it doesn't seem feasible because I can't add a no - arg constructor in my custom writer because IndexWriter
doesn't have any no - arg constructor. 考虑到Q1 , Q2这两个问题,这似乎不可行,因为我无法在自定义IndexWriter
器中添加no-arg构造函数,因为IndexWriter
没有任何no-arg构造函数。
public class CustomIndexWriter extends IndexWriter implements Serializable {
/*
private Directory d;
private IndexWriterConfig conf;
public CustomIndexWriter(){
super();
super(this.d, this.conf);
}
*/
public CustomIndexWriter(Directory d, IndexWriterConfig conf) throws IOException {
super(d, conf);
}
/**
*
*/
private static final long serialVersionUID = 1L;
private void readObject(ObjectInputStream input) throws IOException, ClassNotFoundException{
input.defaultReadObject();
}
private void writeObject(ObjectOutputStream output) throws IOException, ClassNotFoundException {
output.defaultWriteObject();
}
}
In above code, I can't add constructor shown as commented because no - arg constructor doesn't exist in Super class and can't access this
fields before super
. 在上面的代码中,我无法添加显示为注释的构造函数,因为在Super类中不存在no-arg构造函数,并且在super
之前无法访问this
字段。
Is there a way to achieve this? 有没有办法做到这一点?
You can always add a parameter-less constructor. 您始终可以添加无参数的构造函数。
Eg: 例如:
public class CustomWriter extends IndexWriter implements Serializable {
private Directory lDirectory;
private IndexWriterConfig iwConfig;
public CustomWriter() {
super();
// Assign default values
this(new Directory("." + System.getProperty("path.separator")), new IndexWriterConfig());
}
public CustomWriter(Directory dir, IndexWriterConfig iwConf) {
lDirectory = dir;
iwConfig = iwConf;
}
public Directory getDirectory() { return lDirectory; }
public IndexWriterConfig getConfig() { return iwConfig; }
public void setDirectory(Directory dir) { lDirectory = dir; }
public void setConfig(IndexWriterConfig conf) { iwConfig = conf; }
// ...
}
EDIT: 编辑:
Having taken a look at my own code (using Lucene.Net), the IndexWriter needs an analyzer, and a MaxFieldLength. 看完我自己的代码(使用Lucene.Net)之后,IndexWriter需要一个分析器和一个MaxFieldLength。
So the super-call would look something like this: 因此,超级呼叫看起来像这样:
super(new Directory("." + System.getProperty("path.separator")), new StandardAnalyzer(), MaxFieldLength.UNLIMITED);
So adding these values as defaults should fix the issue. 因此,将这些值添加为默认值应该可以解决此问题。 Maybe then add getter- and setter-methods for the analyzer and MaxFieldLength, so you have control over that at a later stage. 也许然后为分析器和MaxFieldLength添加getter和setter方法,这样您就可以在稍后阶段对其进行控制。
I am not sure how but this syntax works in Spring Batch and ExecutionContext
returns a non - null Object in StepExecutionListenerSupport
. 我不确定如何运行,但是该语法在Spring Batch中工作,并且ExecutionContext
在StepExecutionListenerSupport
返回非null对象。
public class CustomIndexWriter implements Serializable {
private static final long serialVersionUID = 1L;
private transient IndexWriter luceneIndexWriter;
public CustomIndexWriter(IndexWriter luceneIndexWriter) {
this.luceneIndexWriter=luceneIndexWriter;
}
public IndexWriter getLuceneIndexWriter() {
return luceneIndexWriter;
}
public void setLuceneIndexWriter(IndexWriter luceneIndexWriter) {
this.luceneIndexWriter = luceneIndexWriter;
}
}
I put an instance of CustomIndexWriter
in step partitioner, partitioned step chunk works with writer by doing, getLuceneIndexWriter()
and then in StepExecutionListenerSupport
, I close this writer. 我将CustomIndexWriter
的实例放在步骤分区器中,分区的步骤块通过执行getLuceneIndexWriter()
与writer一起工作,然后在StepExecutionListenerSupport
关闭该writer。
This way my spring batch partitioned step works with a single instance of Lucene Index Writer Object. 这样,我的春季批处理分区步骤就可以与Lucene Index Writer Object的单个实例一起工作。
I was hoping that I will get a NullPointer if trying to perform operation on writer obtained by getLuceneIndexWriter()
but that doesn't happen ( despite it being transient
). 我希望如果尝试对通过getLuceneIndexWriter()
获得的getLuceneIndexWriter()
执行操作,那么我将获得一个NullPointer,但这不会发生(尽管它是transient
)。 I am not sure why this works but it does. 我不知道为什么会这样,但确实可以。
For Spring Batch job metadata, I am using in - memory repository and not db based repository. 对于Spring Batch作业元数据,我使用的是内存存储库,而不是基于db的存储库。 Not sure if this will continue to work once I start using db for metadata. 不知道一旦我开始使用db作为元数据,这是否将继续有效。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.