String.format slow, need faster alternative

Question

I was hoping to get some advice on how to speed up the following function. Specifically, I'm hoping to find a faster way to convert numbers (mostly doubles, IIRC there's one int in there) to strings to store as Listview subitems. As it stands, this function takes 9 seconds to process 16 orders! Absolutely insane, especially considering that with the exception of the call to process the DateTimes, it's all just string conversion.

I thought it was the actual displaying of the listview items that was slow, so I did some research and found that adding all subitems to an array and using Addrange was far faster than adding the items one at a time. I implemented the change, but got no better speed.

I then added some stopwatches around each line to narrow down exactly what's causing the slowdown; unsurprisingly, the call to the datetime function is the biggest slowdown, but I was surprised to see that the string.format calls were extremely slow as well, and given the number of them, make up the majority of my time.

    private void ProcessOrders(List<MyOrder> myOrders)
    {
        lvItems.Items.Clear();
        marketInfo = new MarketInfo();
        ListViewItem[] myItems = new ListViewItem[myOrders.Count];
        string[] mySubItems = new string[8];
        int counter = 0;
        MarketInfo.GetTime();
        CurrentTime = MarketInfo.CurrentTime;
        DateTime OrderIssueDate = new DateTime();

        foreach (MyOrder myOrder in myOrders)
        {
            string orderIsBuySell = "Buy";
            if (!myOrder.IsBuyOrder)
                orderIsBuySell = "Sell";
            var listItem = new ListViewItem(orderIsBuySell);

            mySubItems[0] = (myOrder.Name);
            mySubItems[1] = (string.Format("{0:g}", myOrder.QuantityRemaining) + "/" + string.Format("{0:g}", myOrder.InitialQuantity));
            mySubItems[2] = (string.Format("{0:f}", myOrder.Price));
            mySubItems[3] = (myOrder.Local);
            if (myOrder.IsBuyOrder)
            {
                if (myOrder.Range == -1)
                    mySubItems[4] = ("Local");
                else
                    mySubItems[4] = (string.Format("{0:g}", myOrder.Range));
            }
            else
                mySubItems[4] = ("N/A");
            mySubItems[5] = (string.Format("{0:g}", myOrder.MinQuantityToBuy));
            string IssueDateString = (myOrder.DateWhenIssued + " " + myOrder.TimeWhenIssued);
            if (DateTime.TryParse(IssueDateString, out OrderIssueDate))
                mySubItems[6] = (string.Format(MarketInfo.ParseTimeData(CurrentTime, OrderIssueDate, myOrder.Duration)));
            else
                mySubItems[6] = "Error getting date";
            mySubItems[7] = (string.Format("{0:g}", myOrder.ID));
            listItem.SubItems.AddRange(mySubItems);
            myItems[counter] = listItem;
            counter++;

        }
        lvItems.BeginUpdate();
        lvItems.Items.AddRange(myItems.ToArray());
        lvItems.EndUpdate();
    }

Here's the time data from a sample run:
0: 166686
1: 264779
2: 273716
3: 136698
4: 587902
5: 368816
6: 955478
7: 128981

Where the numbers are equal to the indexes of the array. All other lines were so low in ticks as to be negligible compared to these.

Although I'd like to be able to use the number formatting of string.format for pretty output, I'd like to be able to load a list of orders within my lifetime more, so if there's an alternative to string.format that's considerably faster but without the bells and whistles, I'm all for it.

---

Edit: Thanks to all of the people who suggested the myOrder class might be using getter methods rather than actually storing the variables as I originally thought. I checked that and sure enough, that was the cause of my slowdown. Although I don't have access to the class to change it, I was able to piggyback onto the method call to populate myOrders and copy each of the variables to a list within the same call, then use that list when populating my Listview. Populates pretty much instantly now. Thanks again.

Answer 1

I find it hard to believe that simple string.Format calls are causing your slowness problems - it's generally a very fast call, especially for nice simple ones like most of yours.

But one thing that might give you a few microseconds...

Replace

string.Format("{0:g}", myOrder.MinQuantityToBuy)

with

myOrder.MinQuantityToBuy.ToString("g")

This will work when you're doing a straight format of a single value, but isn't any good for more complex calls.

Answer 2

I put all the string.format calls into a loop and was able to run them all 1 million times in under a second, so your problem isn't string.format...it's somewhere else in your code.

Perhaps some of these properties have logic in their getter methods? What sort of times do you get if you comment out all the code for the listview?

Answer 3

It is definitely not string.Format that is slowing you down. Suspect the property accesses from myOrder.

On one of the format calls, try to declare a few local variables and set those to the properties you try to format, then pass those local variables to yoru string.Format and retime. You may find that your string.Format now runs in lightning speed as it should.

Now, property accesses usually don't require much time to run. However, I've seen some classes where each property access is logged (for audit trail). Check if this is the case and if some operation is holding your property access from returning immediately.

If there is some operation holding a property access, try to queue up those operations (eg queue up the logging calls) and have a background thread execute them. Return the property access immediately.

Also, never put slow-running code (eg elaborate calculations) into a property accesser/getter, nor code that has side-effects. People using the class will not be aware that it will be slow (since most property accesses are fast) or has side effects (since most property accesses do not have side effects). If the access is slow, rename the access to a GetXXX() function. If it has side effects, name the method something that conveys this fact.

Answer 4

Wow. I feel a little stupid now. I've spent hours beating my head against the wall trying to figure out why a simple string operation would be taking so long. MarketOrders is (I thought) an array of myOrders, which is populated by an explicit call to a method which is severely restricted as far as how times per second it can be run. I don't have access to that code to check, but I had been assuming that myOrders were simple structs with member variables that were assigned when MarketOrders is populated, so the string.format calls would simply be acting on existing data. On reading all of the replies that point to the access of the myOrder data as the culprit, I started thinking about it and realized that MarketOrders is likely just an index, rather than an array, and the myOrder info is being read on demand. So every time I call an operation on one of its variables, I'm calling the slow lookup method, waiting for it to become eligible to run again, returning to my method, calling the next lookup, etc. No wonder it's taking forever.

Thanks for all of the replies. I can't believe that didn't occur to me.

Answer 5

I am glad that you got your issue solved. However I did a small refactoring on your method and came up with this:

    private void ProcessOrders(List<MyOrder> myOrders)
    {
        lvItems.Items.Clear();
        marketInfo = new MarketInfo();
        ListViewItem[] myItems = new ListViewItem[myOrders.Count];
        string[] mySubItems = new string[8];
        int counter = 0;
        MarketInfo.GetTime();
        CurrentTime = MarketInfo.CurrentTime;
        // ReSharper disable TooWideLocalVariableScope
        DateTime orderIssueDate;
        // ReSharper restore TooWideLocalVariableScope

        foreach (MyOrder myOrder in myOrders)
        {
            string orderIsBuySell = myOrder.IsBuyOrder ? "Buy" : "Sell";
            var listItem = new ListViewItem(orderIsBuySell);

            mySubItems[0] = myOrder.Name;
            mySubItems[1] = string.Format("{0:g}/{1:g}", myOrder.QuantityRemaining, myOrder.InitialQuantity);
            mySubItems[2] = myOrder.Price.ToString("f");
            mySubItems[3] = myOrder.Local;

            if (myOrder.IsBuyOrder)
                mySubItems[4] = myOrder.Range == -1 ? "Local" : myOrder.Range.ToString("g");
            else
                mySubItems[4] = "N/A";

            mySubItems[5] = myOrder.MinQuantityToBuy.ToString("g");

            // This code smells:
            string issueDateString = string.Format("{0} {1}", myOrder.DateWhenIssued, myOrder.TimeWhenIssued);
            if (DateTime.TryParse(issueDateString, out orderIssueDate))
                mySubItems[6] = MarketInfo.ParseTimeData(CurrentTime, orderIssueDate, myOrder.Duration);
            else
                mySubItems[6] = "Error getting date";

            mySubItems[7] = myOrder.ID.ToString("g");

            listItem.SubItems.AddRange(mySubItems);
            myItems[counter] = listItem;
            counter++;
        }
        lvItems.BeginUpdate();
        lvItems.Items.AddRange(myItems.ToArray());
        lvItems.EndUpdate();
    }

This method should be further refactored:

1. Remove outer dependencies Inversion of control (IoC) in mind and by using dependency injection (DI);
2. Create new property "DateTimeWhenIssued" for MyOrder that will return DateTime data type. This should be used instead of joining two strings (DateWhenIssued and TimeWhenIssued) and then parsing them into DateTime;
3. Rename ListViewItem as this is a built in class;
4. ListViewItem should have a new constructor for boolean "IsByOrder": var listItem = new ListViewItem(myOrder.IsBuyOrder). Instead of a string "Buy" or "Sell";
5. "mySubItems" string array should be replaced with a class for better readability and extendibility;
6. Lastly, the foreach (MyOrder myOrder in myOrders) could be replaced with a "for" loop as you are using a counter anyway. Besides "for" loops are faster too.

Hopefully you do not mind my suggestions and that they are doable in your situation.

PS. Are you using generic arrays? ListViewItem.SubItems property could be public List<string> SubItems { get; set; } public List<string> SubItems { get; set; } public List<string> SubItems { get; set; } ;