弹性搜索索引

Question

ElasticSearch 为 UI 创建的新记录做索引，但是 liquibase 文件创建的记录没有索引所以不会出现在搜索结果中，ElasticSearch 应该索引 UI 和 liquibase 文件创建的所有记录，是否有索引记录的过程liquibase 文件。

Answer 1

Liquibase only makes changes to your database. Unless you have some process which listens to the database changes and then updates Elasticsearch, you will not see the changes.

There might be multiple ways to get your database records into Elasticsearch:

1. Your UI probably calls some back-end code to index a create or an update into Elasticsearch already
2. Have a batch process which knows which records are changed (eg use an updated flag column or a updated_timestamp column) and then index those into Elasticsearch.

The second option can either be done in code using a scripting or back-end scheduled job or you might be able to use Logstash with the jdbc-input plugin.

Answer 2

As Sarwar Bhuiyan and Mogsdad sad

Unless you have some process which listens to the database changes and then updates Elasticsearch

You can use liquibase to populate elasticsearch(this task will be executed once, just like normal migration). To do this you need to create a customChange :

<customChange class="org.test.ElasticMigrationByEntityName">
    <param name="entityName" value="org.test.TestEntity" />
</customChange>

In that java based migration you can call the services you need. Here is an example of what you can do ( please do not use code from this example in a production ).

public class ElasticMigrationByEntityName implements CustomTaskChange {

    private String entityName;

    public String getEntityName() {
        return entityName;
    }

    public void setEntityName(String entityName) {
        this.entityName = entityName;
    }

    @Override
    public void execute(Database database) {
        //We schedule the task for the next execution. We are waiting for the context to start and we get access to the beans
        DelayedTaskExecutor.add(new DelayedTask(entityName));
    }

    @Override
    public String getConfirmationMessage() {
        return "OK";
    }

    @Override
    public void setUp() throws SetupException {

    }

    @Override
    public void setFileOpener(ResourceAccessor resourceAccessor) {

    }

    @Override
    public ValidationErrors validate(Database database) {
        return new ValidationErrors();
    }

    /* ===================== */

    public static class DelayedTask implements Consumer<ApplicationContext> {

        private final String entityName;

        public DelayedTask(String entityName) {
            this.entityName = entityName;
        }

        @Override
        public void accept(ApplicationContext applicationContext) {
            try {
                checkedAccept(applicationContext);
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }

        //We're going to find beans by name (the most controversial point)
        private void checkedAccept(ApplicationContext context) throws ClassNotFoundException {
            Class entityClass = Class.forName(entityName);
            String name = entityClass.getSimpleName();

            //Please do not use this code in production
            String repositoryName = org.apache.commons.lang3.StringUtils.uncapitalize(name + "Repository");
            String repositorySearchName = org.apache.commons.lang3.StringUtils.uncapitalize(name + "SearchRepository");

            JpaRepository repository = (JpaRepository) context.getBean(repositoryName);
            ElasticsearchRepository searchRepository = (ElasticsearchRepository) context.getBean(repositorySearchName);

            //Doing our work
            updateData(repository, searchRepository);
        }

        //Write your logic here
        private void updateData(JpaRepository repository, ElasticsearchRepository searchRepository) {
            searchRepository.saveAll(repository.findAll());
        }

    }

}

Because the beans have not yet been created, we will have to wait for them

@Component
public class DelayedTaskExecutor {

    @Autowired
    private ApplicationContext context;

    @EventListener
    //We are waiting for the app to launch
    public void onAppReady(ApplicationReadyEvent event) {
        Queue<Consumer<ApplicationContext>> localQueue = getQueue();
        if(localQueue.size() > 0) {
            for (Consumer<ApplicationContext> consumer = localQueue.poll(); consumer != null; consumer = localQueue.poll()) {
                consumer.accept(context);
            }
        }
    }

    public static void add(Consumer<ApplicationContext> consumer) {
        getQueue().add(consumer);
    }

    public static Queue<Consumer<ApplicationContext>> getQueue() {
        return Holder.QUEUE;
    }

    private static class Holder {

        private static final Queue<Consumer<ApplicationContext>> QUEUE = new ConcurrentLinkedQueue();

    }

}

An entity example:

@Entity
@Table(name = "test_entity")
@Document(indexName = "testentity")
public class TestEntity implements Serializable {

    private static final long serialVersionUID = 1L;

    @Id
    @Field(type = FieldType.Keyword)
    @GeneratedValue(generator = "uuid")
    @GenericGenerator(name = "uuid", strategy = "uuid2")
    private String id;

    @NotNull
    @Column(name = "code", nullable = false, unique = true)
    private String code;

    ...

}