[英]Connecting solr with aws RDS Mysql through data import handler
I recently started implementing solr-cloud on AWS EC2 for search applications.我最近开始在 AWS EC2 上为搜索应用程序实施 solr-cloud。 I have created 2 AWS Ec2 instances with the following configurations ---
我创建了 2 个具有以下配置的 AWS Ec2 实例 ---
For the 2 EC2 instances, I have created a security group which allows all inbound traffic.对于 2 个 EC2 实例,我创建了一个允许所有入站流量的安全组。 NACL has default settings that allows all inbound traffic as well.
NACL 具有允许所有入站流量的默认设置。
Steps Followed to install Apache Solr -安装 Apache Solr 的步骤 -
ssh -i "pem_file" ubuntu@ec2-public-ipv4-address
sudo apt-update
sudo apt-update
sudo apt-get openjdk-11
sudo apt-get openjdk-11
wget https://archive.apache.org/dist/lucene/solr/8.3.0/solr-8.3.0.tgz
wget https://archive.apache.org/dist/lucene/solr/8.3.0/solr-8.3.0.tgz
tar -xvzf solr-8.3.0.tgz
tar -xvzf solr-8.3.0.tgz
export SOLR_HOME=/opt/solr-8.3.0
sudo bin/solr start -c -p 8983 -h solr-node-1 -force
sudo bin/solr start -c -p 8983 -h solr-node-1 -force
sudo lsof -i -P -n | grep LISTEN
sudo lsof -i -P -n | grep LISTEN
检查打开的端口sudo lsof -i -P -n | grep LISTEN
bin/solr create -c travasko -d sample_techproducts_configs -n travasko_configs -shards 2 -rf 2 -p 8983
bin/solr create -c travasko -d sample_techproducts_configs -n travasko_configs -shards 2 -rf 2 -p 8983
创建 collections、分片和副本I repeated the same process on the other EC2 machine and ran solr on it.我在另一台 EC2 机器上重复了相同的过程,并在其上运行了 solr。 Now, to use the data import handler in solr, I edited the following files:
现在,为了使用 solr 中的数据导入处理程序,我编辑了以下文件:
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</requestHandler>
<dataConfig>
<dataSource type="JdbcDataSource"
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://examplerds.cuhj86yfdpid.us-east-1.rds.amazonaws.com:3306/TRAVASKODB1"
user="examplerds"
password="examplerds#123"/>
<document>
<entity name="MOMENTS"
pk="MOMENT_ID"
query="SELECT MOMENT_ID,MOMENT_TEXT FROM MOMENTS"
deltaImportQuery="SELECT MOMENT_ID,MOMENT_TEXT FROM MOMENTS WHERE MOMENT_ID='${dih.delta.MOMENT_ID}'"
deltaQuery="SELECT MOMENT_ID FROM MOMENTS WHERE LAST_MODIFIED > '${dih.last_index_time}'"
>
<field column="MOMENT_ID" name="MOMENT_ID"/>
<field column="MOMENT_TEXT" name="MOMENT_TEXT"/>
</entity>
</document>
</dataConfig>
<schema name="MOMENTS" version="1.5">
<field name="_version_" type="long" indexed="true" stored="true"/>
<field name="MOMENT_ID" type="integer" indexed="true" stored="true" required="true" multiValued="false" />
<field name="MOMENT_TEXT" type="string" indexed="true" stored="true" multiValued="false" />
</schema>
wget -q "http://search.maven.org/remotecontent?filepath=mysql/mysql-connector-java/5.1.32/mysql-connector-java-5.1.32.jar" -O mysql-connector-java.jar
<lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" />
<lib dir="${solr.install.dir:../../../..}/dist/" regex="mysql-connector-java.jar" />
bin/solr zk -n travasko_config -z solr-node-1:9983 cp /opt/solr-8.3.0/server/solr/configsets/_default/conf/managed-schema zk:/configs/travasko_config/managed-schema
The solrconfig.xml file for this index does not have an operational DataImportHandler defined!
Note: The AWS RDS and EC2 instances are in the same VPC sharing the same Security Group.注意: AWS RDS 和 EC2 实例位于同一个 VPC 中,共享同一个安全组。
So why is solrconfig.xml file throwing an error during dataimport?那么为什么 solrconfig.xml 文件在数据导入期间会抛出错误? What am i missing here?
我在这里想念什么?
The solution to the above issue was basically setting the java system property for solr versions greater than 8.2.0 as below:上述问题的解决方案基本上是为大于 8.2.0 的 solr 版本设置 java 系统属性,如下所示:
-Denable.dih.dataConfigParam=true
This parameter can be set either in solr.in.cmd or solr.in.sh which can be found inside the directory below: ,此参数可以在 solr.in.cmd 或 solr.in.sh 中设置,可以在以下目录中找到:
/opt/solr-8.3.0/bin
If, /opt/solr-8.3.0 is the installation directory of solr.如果,/opt/solr-8.3.0是solr的安装目录。
The other method was to pass this parameter as command line parameter while starting solr as below:另一种方法是在启动 solr 时将此参数作为命令行参数传递,如下所示:
sudo bin/solr start -c -p 8983 -h solr-node-1 -Denable.dih.dataConfigParam=true -force
solr-node-1 is the public IPv4 address of the AWS Ec2 instance on which solr is configured. solr-node-1 是配置了 solr 的 AWS Ec2 实例的公共 IPv4 地址。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.