Does FIWARE Cygnus have any issue with space usage?

Question

I'm using some FIWARE components (Orion 2.4.0-bext and Cygnus 2.3.0). Also I'm using mongoDB (version 3.6) for current context information and MySQL (version 5.7) for saving historical changes using Cygnus. All component are dockerized, each one on its own container.

In my current solution, I receive lots of messages (between 100 and 500 per second) that are stored into mongoDB (through Orion) and MySQL (through Cygnus). The problem that I found is that

1. Mongo and Cygnus are using a lot of CPU power, even tough the amount of data is not huge.
2. Cygnus is growing insanely in the amount of disk used by the container.

Cygnus has been configured to store 1 row per attribute change in MySQL. These are the docker-compose environment variables of Cygnus regarding the configuration of the mapping in MySQL:

    environment:
      - ...
      - "CYGNUS_LOG_LEVEL=DEBUG" #
      - "CYGNUS_MYSQL_DATA_MODEL=dm-by-entity-type"
      - "CYGNUS_MYSQL_ATTR_NATIVE_TYPES=true"
      - "CYGNUS_MYSQL_ATTR_PERSISTENCE=column"

To ilustrate point 2), I'll show you how much space is using Cygnus after and bofere running for a while.

When I check the disk used:

[root@maaster-docker-test docker]# du -h --max-depth=1

4.0K    ./trust
20K     ./builder
11M     ./image
12G     ./overlay2
4.0K    ./runtimes
72K     ./buildkit
100K    ./network
4.0K    ./tmp
807M    ./volumes
4.0K    ./swarm
20K     ./plugins
38G     ./containers
50G     .

We can see that containers folder is using 38G of space. Inspecting a little more:

[root@maaster-docker-test containers]# du -h --max-depth=1

35G     ./dd1c103dd7d1a5ecbab0e3a2e039a6c4d3fd525837a766a9bab28b29ba923a32
1.4M    ./ebf1c3978077a727d414d3e2de5974b03a236999cc66c29f66c0e517d6bbe055
40K     ./2c92baf1368ee292b7cf33db86369d0b6f7941753f54f145a14e5793eac6eba2
40K     ./047c082e1f1e8f26be0bb1a063e93907f55b4736ad88fd29736e383c6e03d559
175M    ./63c715643cfbd7695dc538081e4a963e270dc03a4ffdb528bb375cf57438a477
152M    ./cbb5fe85b16411dc94c8ab00dcd7b40b728acaa422398445be4130aa0197d287
1.4M    ./e7c01dca2246f17c5f0aa9f413772b7b73be2d962ae772e062c5257ce95252fa
2.6G    ./534b92fe6f1dac9f56d59b1e9feeb0d013199d17fae0af1df150dd20b96c9f70
38G     .

container starting with dd1c103dd7d1 is taking 35G of space. When I check to whom this container corresponds:

[root@maaster-docker-test containers]# docker ps

...
dd1c103dd7d1        fiware/cygnus-ngsi:latest               "/cygnus-entrypoint.…"   2 days ago          Up 2 days (healthy)     0.0.0.0:5050-5051->5050-5051/tcp, 0.0.0.0:5080->5080/tcp   fiware-cygnus
...

Turns out that is the Cygnus. At this point, I had been inserted around 12000 different entities in MongoDB, reflecting around 300.000 changes (rows) in MySQL. So the amount of data is not that insane.

To finalize, after restarting the services defined in docker-compose, this is stopping Cygnus, removing containers and starting them again):

[root@maaster-docker-test containers]# docker ps
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS                     PORTS                                                      NAMES
...
0fb5ab78b70b        fiware/cygnus-ngsi:latest               "/cygnus-entrypoint.…"   3 minutes ago       Up 3 minutes (healthy)     0.0.0.0:5050-5051->5050-5051/tcp, 0.0.0.0:5080->5080/tcp   fiware-cygnus
...

And checking again for space used

[root@maaster-docker-test containers]# du -h --max-depth=1
48K     ./e8d32c563ad0a22f70a83b576592b32b6336b0b39cc401cfe451896e6b76a897
40K     ./043069752e112cdce051d3792b8a261133b869e939bed263b9e0eb7dc265c379
80K     ./f1f40e29a8d620fc38b7b12cfcd4986f3ffcc2f970f08d582cb826c2f2a41ff0
1.1M    ./0fb5ab78b70bcc856c91b9b99266049b20f05f5d1d6d7d224cdfde9eec47b801
40K     ./047c082e1f1e8f26be0bb1a063e93907f55b4736ad88fd29736e383c6e03d559
128K    ./aa3c63cafe57c6ea1b4ed01bee9415b6a87bf538d6d7a700750be21c738d39d5
68K     ./11388b7090bcd08ac0235129b7354e5c9b68f1d4855abdc97f025ad597b73e68
68K     ./84a6aaf4221fde5cb0e21b727e33e21ff58d275597b121ba22f87296148b38db
1.6M    .

We can see that the container used by Cygnus ( 0fb5ab78b70b ), is taking only 1.1M of space.

So, does anyone know if there is a problem with Cygnus, with Docker, or with Cygnus running in a Docker environment? Is there any configuration change I should perform on Cygnus to be more efficient and also consume less disk space?

Also, I'd be nice to know if there is any configuration change I must to do mongoDB (or maybe ORION) in order to make mongodb more efficient? I don't think that inserting ~200 new entities per second should not impact in the CPU usage of mongo process.

Thanks a lot in advance

Answer 1

I've discovered that the configuration

"CYGNUS_LOG_LEVEL=DEBUG" 

was generating an enormous amount of data that was putting down the performance of the service, increasing the disk usage.

Changing that value solved my issue.