Relation of between CPU vCores and spark executor-cores

Question

I am launching an EMR cluster with m5.xlarge core nodes as follow:

4 vCore, 16 GiB memory, EBS only storage
EBS Storage:200 GiB

I am not sure if I run an Spark Job and setting the executor-core parameter such as 2. If this is exactly the same as vCores in my m5.xlarge instances?

Answer 1

each executor's core will use a vCore in your cluster, hence create either 4 executors with 1 core each or 2 executors with 2 cores each or 1 executor with 4 cores. Enjoy!

Answer 2

AWS defines 1 core as 1vCPU for most instances. I believe that's the same as vCore

"“Each vCPU is a hyper-thread of an Intel Xeon core except for T2.”

In other words, for T2 instances, 1 vCPU = 1 physical core. For all others, 1 vCPU = 1 logical core. This can lead to significant performance differences when it comes to multi-threaded, burst CPU usage."

Reference: https://www.credera.com/insights/whats-in-a-vcpu-state-of-amazon-ec2-in-2018#:~:text=%E2%80%9CEach%20vCPU%20is%20a%20hyper,%2Dthreaded%2C%20burst%20CPU%20usage .