How is nice cpu percentage calculated, e.g. in top?

Question

My research group is sharing time on a CentOS server, and we've been using renice +15 to try to lower the priority of long-running background tasks. When running top, these processes do show up as having a nice value of 15, but the "%ni" measure of nice cpu load is always very low (less than 1%) even when these processes are churning along on 30 cores (as reported in the %CPU column). This has made us think that we are not actually using renice correctly (though the nice processes do seem to yield to higher-priority tasks). How exactly is the nice cpu percentage calculated in top?

Answer 1

The numbers in top come from reading the file /proc/stat. The first line contains a summary for all the cpus combined. The first column is usr time, the second nice time. These times are in clock ticks, usually 100 per second, and are cumulative, so you have to look over and interval and subtract the start value from the end value. You can see the docs for more details, I like http://man7.org/linux/man-pages/man5/proc.5.html

The Linux kernel adds CPU time to the nice column if the nice value is greater than 0, otherwise it puts it in the usr column.

The nice value for an individual process can be found by looking at column 19 of /proc/[pid]/stat. That number should be 15 for you, and the number in column 18 should be 35 (The kernel's internal interpretation of a nice of 15.) However, if top is showing those as 15 in the NI column, it is getting that value from /proc/[pid]/stat.

Comparing the CPU time used in usr and sys in /proc/[pid]/stat and then usr, nice and sys in /proc/stat will give you a good idea of where the time is going. Maybe there are just a ton of CPUs on the system.