C＃应用程序配置文件给出不同的结果

Question

I'm new to profiling. I'm trying to profile a C# application which connects to an SQLite database and retrieve data. The database contains 146856400 rows and the select query retrieves 428800 rows after execution.

On the first execution the main thread takes 246686 ms

On second execution of the same code the main thread takes only 4296 ms

After restarting the system

On the first execution the main thread takes 244533 ms

On the second execution of the same code the main thread takes only 4053 ms

Questions:

1) Why is there a big difference between the first execution timing and the second execution timing

2) After restarting the system why I'm not getting the same results.

Pls help

Answer 1

You experience the difference between cold and warm execution of your query. Cold means the first time and warm all subsequent invocations of your db query. The first time everything is "cold"

• OS file system cache is empty.
• SQLLite cache is empty.
• ORM dynamic query compilation is not done and cached yet.
• ORM Mapper cache is empty.
• Garbage Collector needs to tune your working set
• ....

When you execute your query a second time all these first time initializations (caching) are done and you are measuring the effects of different cache levels as long as there is enough memory available to cache a substantial amount of your requested data.

A performance difference between 4 minutes and 4s is impressive. Both numbers are valid. Measuring something is easy. Telling someone else what exactly you have measured and how the performance can be improved by changing this or that is much harder.

The performance game goes often like this:

Customer: It is slow 
Dev:      I cannot repro your issue.
Customer: Here is my scenario .... 
Dev:      I still cannot repro it. Can you give me data set you use and the exact steps you did perform?
Customer: Sure. Here is the data and the test steps.
Dev:      Ahh I see. I can make it 10 times faster.
Customer: That is great. Can I have the fix?
Dev:      Sure here it is.
Customer: **Very Angry** It has become faster yes. But I cannot read my old data!
Dev:      Ups. We need to migrate all your old data to the new much more efficient format. 
          We need to  develop a a conversion tool which will take 3 weeks and your site will 
          have 3 days downtime while the conversion tool is running. 
          Or 
          We keep the old inefficient data format. But then we can make it only 9 times faster.
Customer: I want to access my data faster without data conversion!
Dev:      Here is the fix which is 10% slower with no schema changes. 
Customer: Finally. The fix does not break anything but it has not become faster?
Dev:      I have measured your use case. It is only slow for the first time. 
          All later data retrievals are 9 times faster than before. 
Customer: Did I mention that in my use case I read always different data?
Dev:      No you did not. 
Customer: Fix it!
Dev:      That is not really possible without a major rewrite of large portions of our software.
Customer: The data I want to access is stored in a list. I want to process it sequentially.
Dev:      In that case we can preload the data in the background while you are working the current data set. You will only experience a delay for the first data set on each working day.
Customer: Can I have the fix?
Dev:      Sure here it is.
Customer: Perfect. It works!

Performance is hard to grasp since most of the time you deal with perceived performance which is subjective. Bringing it down to quantitative measurements is a good start but you need to tune your metrics to reflect actual customer use cases or you will likely optimize at the wrong places like above. A complete understanding of customer requirements and use cases is a must. On the other hand you need to understand your complete system (profile it as hell) to be able to tell the difference between cold and warm query execution and where you can tune the whole thing. These caches become useless if you query for different data all of the time (not likely). Perhaps you need a different index to speed up queries or you buy a SSD or you keep all of the data in memory and do all subsequent queries in memory....