[英]Adding Geo_shape to Elasticsearch using Logstash
我有一個CSV文件,其中包含WKT格式的幾何圖形。 我試圖使用CSV文件提取geo_shape數據。 我創建了文件“ input_mapping.json”中給出的映射
{
"mappings" : {
"doc" : {
"properties" : {
"Lot" : {
"type" : "long"
},
"Lot_plan" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Parcel_Address_Line_1" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Plan" : {
"type" : "long"
},
"Tenure" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"WKT" : {
"type" : "geo_shape"
}
}
}
}
}
WKT是我的geo_shape,它是WKT(String)格式。 以下是我嘗試使用logstash插入的輸入CSV文件:
WKT,Lot_plan,Tenure,Parcel_Address_Line_1,Lot,Plan
"POLYGON ((148.41503356 -26.62829003,148.44798048 -26.62800857,148.45234634 -26.63457929,148.45507096 -26.64778132,148.41735984 -26.64808729,148.41514107 -26.64091476,148.41503356 -26.62829003))",21MM1,FH,MASSEY DOWNS,21,1
"POLYGON ((148.45507096 -26.64778132,148.45779641 -26.66098396,148.45859297 -26.66259081,148.45801376 -26.66410383,148.45989472 -26.67278979,148.42510081 -26.67310328,148.42434355 -26.67065659,148.41735984 -26.64808729,148.45507096 -26.64778132))",21MM2,FH,,21,2
"POLYGON ((148.39514404 -26.68791317,148.37228669 -26.68894235,148.37188338 -26.68895271,148.37092744 -26.68897445,148.37051869 -26.68898023,148.36312088 -26.68908468,148.36261958 -26.66909425,148.39598678 -26.66869309,148.39584372 -26.66934742,148.39583604 -26.66968184,148.39590526 -26.67007957,148.39598629 -26.67039933,148.39614586 -26.67085156,148.39625052 -26.67085085,148.42434355 -26.67065659,148.42510081 -26.67310328,148.42537156 -26.67397795,148.42549108 -26.68541445,148.41781484 -26.68547248,148.39988482 -26.68562107,148.39966009 -26.68562292,148.39704234 -26.68564442,148.39514404 -26.68791317))",21MM3,LL,DERWENT PARK,21,3
我的logstash conf文件是:
input{
file{
path=>"D:/input.csv"
start_position=>"beginning"
sincedb_path=>"D:/sample.text"
}
}
filter{
csv{
separator =>","
columns =>["WKT","Lot_plan","Tenure","Parcel_Address_Line_1","Lot","Plan"]
skip_header=>true
skip_empty_columns=>true
convert => {
"Lot" => "integer"
"Plan" => "integer"
}
remove_field =>[ "_source","message","host","path","@version","@timestamp" ]
}
}
output{
elasticsearch{
hosts=>"http://localhost:9701"
index=>"input_mapping"
template =>"D:/input_mapping.json"
template_name => "input_mapping"
manage_template => true
}
}
由於某種原因,它沒有被ElasticSearch攝取。 我正在使用ElasticSearch 6.5.4版和logstash 6.5.4版。 請讓我知道我是否錯過了任何事情。
我意識到會有很多其他開發人員正在尋找與我所遇到過的相似的問題。 稍后,我檢查了提供ElasticSearch攝取的GDAL(ogr2ogr)。 另外,我使用PostgreSQL提取CSV文件。 因此,按照以下步驟使用ogr2ogr工具可以幫助我:
在表中創建另一列,並使用ST_GeomFromText函數更新此列。
更新TableName集WKT_GEOM = ST_GeomFromText(“ WKT”,4632)
(注意:我已經在PostgreSQL中安裝了postgis)
按照提供的示例使用ogr2ogr:
首先使用ogr2ogr創建elasticsearch映射。
b。現在將數據從PostgreSQL接收到ElasticSearch。
這樣,我就能在Elasticsearch中執行地理查詢。 但是不幸的是它沒有logstash。 :(
如果您有任何疑問,請發表評論。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.