使用Cygnus將數據從Orion傳輸到Cosmos時，文件夾和文件都被賦予“空”名稱

Question

我有一個與ngsi2cosmos數據流有關的問題。 將Orion中接收到的信息保存到Cosmos的公共實例中后，一切正常，但是目標文件夾和文件名均為“ null”。

簡單測試如下：

• 我創建了一個全新的NGSIEntity，並添加了以下標頭：Fiware-Service：myservice和Fiware-ServicePath：/ my
• 我添加了一個以Cygnus作為參考終結點的新訂閱。
• 我將更新發送到先前創建的NGSIEntity

在Cosmos中檢查用戶空間時，我檢查是否已創建以下路由：/user/myuser/myservice/null/null.txt

文件內容正常，Orion中的每個更新信息均已正確接收到其中。 問題出在文件夾和文件名上。 我無法使其正常運行。 它不是應該為文件夾和文件命名獲取EntityId和entityType嗎？

組件版本：

• Orion版本：contextBroker-0.19.0-1.x86_64
• 天鵝座版本：cygnus-0.5-91.g3eb100e.x86_64
• 宇宙：全球實例

Cygnus conf文件：

cygnusagent.sources = http-source
cygnusagent.sinks = hdfs-sink 
cygnusagent.channels = hdfs-channel

#=============================================
# source configuration
# channel name where to write the notification events
cygnusagent.sources.http-source.channels = hdfs-channel
# source class, must not be changed
cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
# listening port the Flume source will use for receiving incoming notifications
cygnusagent.sources.http-source.port = 5050
# Flume handler that will parse the notifications, must not be changed
cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
# URL target
cygnusagent.sources.http-source.handler.notification_target = /notify
# Default organization (organization semantic depend on the persistence sink)
cygnusagent.sources.http-source.handler.default_organization = org42
# Number of channel re-injection retries before a Flume event is definitely discarded
cygnusagent.sources.http-source.handler.events_ttl = 10
# Management interface port (FIXME: temporal location for this parameter)
cygnusagent.sources.http-source.handler.management_port = 8081
# Source interceptors, do not change
cygnusagent.sources.http-source.interceptors = ts
# Timestamp interceptor, do not change
cygnusagent.sources.http-source.interceptors.ts.type = timestamp
# Destination extractor interceptor, do not change
cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwreconnectors.cygnus.interceptors.DestinationExtractor$Builder
# Matching table for the destination extractor interceptor, do not change
cygnusagent.sources.http-source.interceptors.de.matching_table = matching_table.conf

# ============================================
# OrionHDFSSink configuration
# channel name from where to read notification events
cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
# sink class, must not be changed
cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
# Comma-separated list of FQDN/IP address regarding the Cosmos Namenode endpoints
cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
# port of the Cosmos service listening for persistence operations; 14000 for httpfs, 50070 for webhdfs and free choice for inifinty
cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
# default username allowed to write in HDFS
cygnusagent.sinks.hdfs-sink.cosmos_default_username = myuser
# default password for the default username
cygnusagent.sinks.hdfs-sink.cosmos_default_password = mypassword
# HDFS backend type (webhdfs, httpfs or infinity)
cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
# how the attributes are stored, either per row either per column (row, column)
cygnusagent.sinks.hdfs-sink.attr_persistence = column
# prefix for the database and table names, empty if no prefix is desired
cygnusagent.sinks.hdfs-sink.naming_prefix =
# Hive FQDN/IP address of the Hive server
cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
# Hive port for Hive external table provisioning
cygnusagent.sinks.hdfs-sink.hive_port = 10000

#=============================================
# hdfs-channel configuration
# channel type (must not be changed)
cygnusagent.channels.hdfs-channel.type = memory
# capacity of the channel
cygnusagent.channels.hdfs-channel.capacity = 1000
# amount of bytes that can be sent per transaction
cygnusagent.channels.hdfs-channel.transactionCapacity = 100

Answer 1

我認為您應該配置cygnus的matching_table來定義路徑和文件名。

該文件位於Cygnus代理程序conf文件的相同路徑中。

您可以按照下一個示例：

# integer id|comma-separated fields|regex to be applied to the fields concatenation|destination|dataset
#
# The available "dictionary" of fields is:
#  - entitydId
#  - entityType
#  - servicePath

1|entityId,entityType|Room\.(\d*)Room|numeric_rooms|rooms