[英]FIWARE QuantumLeap Sanity Check failed: QuantumLeap can't get changed data from Orion
I checked quantumLeap Sanity Check in both virtualMachine(virtualBox with Ubuntu) and production server(CentOS).我在 virtualMachine(带有 Ubuntu 的virtualBox)和生产服务器(CentOS)中检查了quantumLeap Sanity Check。 With the same docker-compose.yml, the sanity check is sucessfully on virtualManchine, but failed on Production server, it's really strange, could anyone can help me?
使用相同的 docker-compose.yml,在 virtualManchine 上的完整性检查成功,但在生产服务器上失败,这真的很奇怪,谁能帮助我? Thank you very much.
非常感谢。 @Jason Fox
@杰森福克斯
The steps of sanity check is followed as https://quantumleap.readthedocs.io/en/latest/admin/check/ , I pasted below.健全性检查的步骤如下https://quantumleap.readthedocs.io/en/latest/admin/check/ ,我粘贴在下面。
The difference of results is only at step 7. On virtualMachine , I can get the updated data from quantumLeap:结果的差异仅在第 7 步。在 virtualMachine 上,我可以从quantumLeap 获取更新的数据:
{
"attrName": "precipitation",
"entityId": "air_quality_observer_be_001",
"index": [
"2020-05-03T11:18:14.000",
"2020-05-03T11:18:55.000"
],
"values": [
0.0,
100.0
]
}
But on production server , the result is:但是在生产服务器上,结果是:
{
"description": "No records were found for such query.",
"error": "Not Found"
}
Note:笔记:
These commands in the sanity check steps are copied to terminal, so there will be no typos.完整性检查步骤中的这些命令被复制到终端,因此不会有拼写错误。
the difference of result is only at step 7. I deleted the old images on both virtualMachine and production server to let the latested images are pulled.结果的差异仅在第 7 步。我删除了 virtualMachine 和生产服务器上的旧图像,以便提取最新的图像。
There was a fiware-base system(orion,mongoDB,IoTAgents,quantumLeap,CrateDB,Grafana) runing on production server and everything was fine.在生产服务器上运行了一个基于固件的系统(orion、mongoDB、IoTAgents、quantumLeap、CrateDB、Grafana),一切都很好。 But one week ago, the hard disk on the server was full and exhausted, and all containers are down.
但是一周前,服务器上的硬盘已经满了,耗尽了,所有的容器都宕机了。 I cleaned some huge log files and free some space, then I bring up the fiware-base system and found that there is no data shown on Grafana.
我清理了一些巨大的日志文件并释放了一些空间,然后我打开了基于固件的系统,发现 Grafana 上没有显示数据。 After checked, I found the reason is QuantumLeap did not store data into crateDB, but I'm not sure the root cause is orion or quantumLeap.
经过检查,我发现原因是QuantumLeap没有将数据存储到crateDB中,但我不确定根本原因是orion还是quantumLeap。 So I decide to do the sanity check as the quantumLeap documents suggested.
因此,我决定按照quantumLeap 文件的建议进行完整性检查。 That's the story.
这就是故事。
BTW, the subscription of quantumLeap can be get from orion, but I can't get the changed data from quantumLeap.顺便说一句,quantumLeap 的订阅可以从 orion 获取,但我无法从quantumLeap 获取更改的数据。 Why the newest changed data is not synchronized to quantumLeap?
为什么最新更改的数据没有同步到quantumLeap?
Sanity check steps:健全性检查步骤:
1.Check Orion version 1.查看猎户座版本
curl -X GET http://0.0.0.0:1026/version -H 'Accept: application/json'
2.Check QuantumLeap version 2.查看QuantumLeap版本
curl -X GET http://0.0.0.0:8668/version -H 'Accept: application/json'
3.Create an Orion Subscription for "QuantumLeap" 3.为“QuantumLeap”创建 Orion 订阅
curl -X POST \
'http://0.0.0.0:8668/v2/subscribe?orionUrl=http://orion:1026/v2&quantumleapUrl=http://quantumleap:8668/v2&entityType=AirQualityObserved' \
-H 'Accept: application/json'
4.Check you cat get such subscription from Orion 4.检查你的猫从猎户座获得这样的订阅
curl -X GET http://0.0.0.0:1026/v2/subscriptions \
-H 'Accept: application/json'
5.Insert an entity of AirQualityObserved into Orion 5.将AirQualityObserved的实体插入Orion
curl -X POST \
'http://0.0.0.0:1026/v2/entities?options=keyValues' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"id": "air_quality_observer_be_001",
"type": "AirQualityObserved",
"address": {
"streetAddress": "IJzerlaan",
"postOfficeBoxNumber": "18",
"addressLocality": "Antwerpen",
"addressCountry": "BE"
},
"dateObserved": "2017-11-03T12:37:23.734827",
"source": "http://testing.data.from.smartsdk",
"precipitation": 0,
"relativeHumidity": 0.54,
"temperature": 12.2,
"windDirection": 186,
"windSpeed": 0.64,
"airQualityLevel": "moderate",
"airQualityIndex": 65,
"reliability": 0.7,
"CO": 500,
"NO": 45,
"NO2": 69,
"NOx": 139,
"SO2": 11,
"CO_Level": "moderate",
"refPointOfInterest": "null"
}'
6.Update the precipitation value of the same entity in Orion. 6.更新猎户座同一实体的降水量值。
curl -X PATCH \
http://0.0.0.0:1026/v2/entities/air_quality_observer_be_001/attrs \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"precipitation": {
"value": 100,
"type": "Number"
}
}'
7.Query the changed record of precipitation from quantumLeap for the same entity. 7.从quantumLeap查询同一实体的降水变化记录。
curl -X GET \
'http://0.0.0.0:8668/v2/entities/air_quality_observer_be_001/attrs/precipitation?type=AirQualityObserved' \
-H 'Accept: application/json'
Parts of docker-compose.yml: docker-compose.yml的零件:
orion:
image: fiware/orion
hostname: orion
container_name: fiware-orion
depends_on:
- mongo-db
expose:
- "1026"
ports:
- "1026:1026"
networks:
- default
command: -dbhost mongo-db -logLevel ERROR -corsOrigin __ALL
mongo-db:
image: mongo:3.6
hostname: mongo-db
container_name: db-mongo
expose:
- "27017"
ports:
- "27017:27017"
networks:
- default
command: --bind_ip_all --smallfiles
volumes:
- mongo-db:/data
cratedb:
image: crate:3.1.2
hostname: cratedb
container_name: db-crate
expose:
- "4200"
- "4300"
- "5432"
ports:
- "4200:4200"
- "4300:4300"
- "5432:5432"
networks:
- default
command: -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*"
volumes:
- crate-db:/data
quantumleap:
image: smartsdk/quantumleap
hostname: quantumleap
container_name: fiware-quantumleap
expose:
- "8668"
ports:
- "8668:8668"
depends_on:
- cratedb
environment:
- CRATE_HOST=cratedb # host name of CrateDB
NOT solved, but have a little clue.没有解决,但有一点线索。
I got some log from quantumLeap.我从quantumLeap 那里得到了一些日志。 There is a crate client in quantumLeap, and the reason may be that the crate client in quantumLeap is not working properly.
quantumLeap 中有一个 crate 客户端,原因可能是quantumLeap 中的 crate 客户端工作不正常。
I paste the log here:我在这里粘贴日志:
crate.client.exceptions.ProgrammingError: SQLActionException[ClusterBlockException: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]
172.18.1.1 - - [04/May/2020 17:00:33] "POST /v2/notify HTTP/1.1" 500 -
INFO:werkzeug:172.18.1.1 - - [04/May/2020 17:00:33] "POST /v2/notify HTTP/1.1" 500 -
INFO:translators.factory:Backend selected for tenant 'iothouse' is: crate
ERROR:app:Exception on /v2/notify [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.6/site-packages/connexion/decorators/decorator.py", line 48, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/site-packages/connexion/decorators/uri_parsing.py", line 143, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/site-packages/connexion/decorators/validation.py", line 172, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/site-packages/connexion/decorators/validation.py", line 347, in wrapper
return function(request)
File "/usr/local/lib/python3.6/site-packages/connexion/decorators/parameter.py", line 126, in wrapper
return function(**kwargs)
File "/src/ngsi-timeseries-api/src/reporter/reporter.py", line 189, in notify
trans.insert(payload, fiware_s, fiware_sp)
File "/src/ngsi-timeseries-api/src/translators/crate.py", line 189, in insert
fiware_servicepath)
File "/src/ngsi-timeseries-api/src/translators/crate.py", line 297, in _insert_entities_of_type
self.cursor.executemany(stmt, entries)
File "/usr/local/lib/python3.6/site-packages/crate/client/cursor.py", line 67, in executemany
self.execute(sql, bulk_parameters=seq_of_parameters)
File "/usr/local/lib/python3.6/site-packages/crate/client/cursor.py", line 54, in execute
bulk_parameters)
File "/usr/local/lib/python3.6/site-packages/crate/client/http.py", line 328, in sql
content = self._json_request('POST', self.path, data=data)
File "/usr/local/lib/python3.6/site-packages/crate/client/http.py", line 448, in _json_request
_raise_for_status(response)
File "/usr/local/lib/python3.6/site-packages/crate/client/http.py", line 187, in _raise_for_status
error_trace=error_trace)
Solved.解决了。 The root cause is at crateDB.
根本原因在于 crateDB。 When the hard disk exausted, the crateDB is down and every table are set with readonly.
当硬盘耗尽时,crateDB 关闭,每个表都设置为只读。 The read-only blocks are not automatically removed from the tables even after the disk space is freed and the threshold is undershot.
即使在磁盘空间被释放并且低于阈值之后,只读块也不会自动从表中删除。
Everything is OK after I set the readonly flag to false with the following command in crateDB:在 crateDB 中使用以下命令将 readonly 标志设置为 false 后一切正常:
SHOW CREATE TABLE <tableName>;
ALTER TABLE <tableName> SET ("blocks.read_only_allow_delete" = FALSE);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.