运行 MR/TEZ/SPARK 作业的纱线 ui 中的队列百分比和集群差异百分比
[英]% of Queue and % of Cluster difference in yarn ui for MR/TEZ/SPARK jobs run
问题描述
While debugging the TEZ/SPARK/MR jobs I can see the below jobs details in YARN UI.
在调试 TEZ/SPARK/MR 作业时，我可以在 YARN UI 中看到以下作业详细信息。
does any one please help me with % of Queue & % of Cluster stats, since it showing 1815% usage?有人请帮我处理 % of Queue 和 % of Cluster stats，因为它显示 1815% 的使用率？
1 个解决方案
解决方案1
0 2021-02-11 14:15:43
% of cluster is stat, that is telling you how much resources does your job take from cluster.集群的百分比是 stat，它告诉您您的工作从集群中占用了多少资源。
% of queue tells you, how much of Absolute Configured Capacity does your query take.队列百分比告诉您，您的查询占用了多少绝对配置容量。
Example:例子：
your cluster has 1000GB of memory and two queues.您的集群有 1000GB 的 memory 和两个队列。 QueueA has Absolute configured capacity of 1%, with max capacity of 100%. QueueA 的绝对配置容量为 1%，最大容量为 100%。 QueueB has Absolute configured capacity of 99% with max capacity of 100%. QueueB 的绝对配置容量为 99%，最大容量为 100%。
If you run query in queueA, you are guaranteed you get at least 10GB of memory, but depending on the cluster load, you can get up to 1000GB of memory.如果在 queueA 中运行查询，则可以保证至少获得 10GB 的 memory，但根据集群负载，您最多可以获得 1000GB 的 memory。
That would mean that when you run query in queueA and you get 20GB of memory, you will be using 200% of the queue, but 2% of the cluster.这意味着当您在 queueA 中运行查询并获得 20GB 的 memory 时，您将使用 200% 的队列，但 2% 的集群。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.