Erlang NUMA Technology configuration

Question

I am trying to run erlang application on openstack vm and getting very poor performance and after testing i found something going on with NUMA, This is what i observe in my test.

My openstack compute host with 32 core so i have created 30 vCPU core vm on it which has all NUMA awareness, when i am running Erlang application benchmark on this VM getting worst performance but then i create new VM with 16 vCPU core (In this case my all VM cpu pinned with Numa-0 node) and in this case benchmark result was great.

based on above test its clear if i keep VM on single numa node then performance is much better but when i spread it out to multiple numa zone it get worse.

But interesting thing is when i run same erlang application run on bare metal then performance is really good, so trying to understand why same application running on VM doesn't perform well?

Is there any setting in erlang to better fit with NUMA when running on virtual machine?

Answer 1

It's possible that Erlang is not able to properly detect the cpu topology of your VM.

You can inspect the cpu topology as seen by the VM using lscpu and lstopo-no-graphics from the hwloc package:

#lscpu | egrep '^(CPU\(s\)|Thread|Core|Socket|NUMA)'
#lstopo-no-graphics --no-io

If it doesn't look correct, consider rebuilding the VM using OpenStack options like hw:cpu_treads=2 hw:cpu_sockets=2 as described at https://specs.openstack.org/openstack/nova-specs/specs/juno/implemented/virt-driver-vcpu-topology.html

On the Erlang side, you might experiment with the Erlang VM options +sct , +sbt as described at http://erlang.org/doc/man/erl.html#+sbt