Entities in FIWARE orion disappear after some time passes

Question

I created a project from https://fiware-tutorials.readthedocs.io/en/latest/time-series-data.html tutorial and just changed the entities name and type and everything work right. But after some time (usually a day) all entities in Orion disappears (although the data in Quantumleap persists) and I can not get the entities properties with this command:

curl -X GET \
  --url 'http://localhost:1026/v2/entities?type=Temp'

What is the problem? is there some restriction in tutorial projects?

Answer 1

The tutorials have been written as an introduction to NGSI, not as a robust architectural solution. The idea is just to get something "quick and dirty" up and running on a developer's machine and various shortcuts have been taken. Indeed the docker-compose files all hold the following disclaimer:

WARNING: Do not deploy this tutorial configuration directly to a production environment

The tutorial docker-compose files have not been written for production deployment and will not scale. A proper architecture has been sacrificed to keep the narrative focused on the learning goals, they are just used to deploy everything onto a single Docker machine. All FIWARE components are running at full debug and extra ports have been exposed to allow for direct calls to services. They also contain various obvious security flaws - passwords in plain text, no load balancing, no use of HTTPS and so on.

This is all to avoid the need of multiple machines, generating certificates, encrypting secrets and so on, purely so that a single docker-compose file can be read as an example to build on, not use directly.

When deploying to a production environment, please refer to the Helm Repository for FIWARE Components in order to scale up to a proper architecture:

see: https://github.com/FIWARE/helm-charts/

Perhaps the most relevant factor here to answer your question, there is typically no Volume Persistence - the tutorials clean up after themselves where possible to avoid leaving data on a user's machine unnecessarily.

If you have lost all your entity data when connecting to Orion, my guess here is that the MongoDB database has exited and restarted for some reason. Since there is deliberately no persistent volume set up, this would mean that all previous entities are lost on the restart.

A solution on how to persist volumes and fix this behaviour can be found in answers to another question on this site - something like:

    version: "3.9"
    services:
      mongodb:
      image: mongo:4.4
      ports:
        - 27017:27017
      volumes:
        - type: volume
          source: mongodb_data_volume
          target: /data/db
    volumes:
      mongodb_data_volume:
        external: true