How to calculate the performance of a algorithm on a high-performance machine through a machine whose performance is low in performance testing?

Question

For Example, If I want to calculate the performance of an recommendation algorithm, I test it on a linux machine which has twos 4G internal storage. And the result is: response time-40ms, cpu load: 2, the number of virtual users is 20, RAM consumption is 70%. So what is the performance of this algorithm on a linux machine which has 4 cores and 8G internal storage(or the 8 cores and 16G storage and ...) when its load is 4? PS:the OS is "Red Hat Enterprise Linux Server release 5.7 (Tikanga)", and the program is running on a jvm. The OS and the operating environment of double machine is the same. I know ideally when the load is 4, the performance is double and the result is the same when load is 2, but in fact the result is different. So, the question is: If we know the follow parameters:

cpu load: x

response time: y ms

number of virtual users: z(the concurrency)

RAM consumption: m%

of an algorithm on a low performance machine, can we calculate its performance(or the tps) on a high performance machine? If we can, how? If we can't, is it because the parameters are not enough or the result is varied.

To simplify the question, I'd like to consider two extreme situations. Firstly, the application is just computing, thats to say, the algorithm only compute data, so it is only related with the Cpu(the load and cpu may double be high). The second situation is that, the algorithm only read data form web, that means it doesn't compute anything(the load may be high, but cpu is low).

Answer 1

Generally speaking, you can't. Not really. And yes, mostly this is because what we experience as "performance" does depend on the environment in a very complex manner.

But there is hope: In some cases, you understand how the software is working and you experience that there is a clear bottleneck for the hardware you are running on. In this case you can, in a very limited range, derive an educated guess about the performance of your software on a "bigger" machine, when the "bigger" is from adding bottleneck resources. The guess will work until you extend the bottleneck resource so much that it is not longer the bottleneck. Usually, another resource will quietly take over then.

Let's, for example, consider the very simple case of a CPU-bound algorithm, eg compiling PhantomJS on a 2 core machine with slow hard disk. Your metering tool shows you 2 cores at 95+% and 1 hour overall runtime. When you know that parallelization will extend to more cores, then you can safely expect 30 min runtime on a 4 core machine, regardless whether you add a petabyte of memory and superfast SSD or not. The situation might change, however, when you add many cores, eg like the 80 on my production machine. Either the compiler might not effectively parallelize, or your disks will not be able to read and write the files quick enough to keep all cores under fire. SSDs might help then. Or you try to add more memory. But will that support to your compile? You can only guess, unless you have very thorough knowledge about the full stack of software you employ . And your stack will very likely provide much surprise for you when you dive in.

From the question, we know far too little about your case, but it seems that you were expecting that your jobs are single-threaded and purely CPU bound. In this case you would expect perfect scaling in number of users or single core performance until you hit the ceiling for the given hardware. Seemingly it was not and so your job is it to find out where the bottlenecks are. It's one of the most complex tasks in software development and it is tightly interwoven with performance tuning, because once you have understood where the bottleneck is, you might either conclude what resource to extend next or you can optimize your software to make use of that resource more careful.

The later extension of your question (regarding CPU and/or web) shows that you are already thinking in the right direction.