Retrieving data from a database in another thread(Unity3D)

Question

I currently have a code which retrieves data from a database and visualizes it in unity3D. However, everytime it retrieves data in the FixedUpdate() function, it spikes dramatically every 1 second. I'm thinking about using threading to do this but i'm not sure what i'm doing wrong.

This is the Function i call in the thread.

 public void retrievefromDB(){
                            if (timeStep - prevTimeStep > 99) {
                                    timeStep -= 1; //special for this dataset
   query = "SELECT * FROM GridData2 WHERE timestep=" + timeStep;

                                    if (showParent)
                                            query += " AND (Level != 10)";
                                    else
                                            query += " AND (Level == 10)";

      query += " AND temperature >= " + minTemp + " AND temperature <= " + maxTemp;
                                    dt.Rows.Clear ();
                                    dt = sqlDB.ExecuteQuery (query);

                                    prevTimeStep = timeStep;
                            }

            }

This code lags the scene every 1 second therefore i tried to put it into a thread.

void FixedUpdate()
    {
    Thread testthread = new Thread(new ThreadStart(retrievefromDB));
        testthread.Start ();
}

After putting it in a thread, it keeps crashing the scene after awhile. Can anyone tell me what I did wrongly? And how do i solve it?

Answer 1

The cause of your original issue is relatively obvious: database access is slow. If you put a database call inline in the FixedUpdate method, you're going to essentially pause your game's movement while the DB access happens (which may well take a second if you have to initialise a connection, for example).

The main issue with your threaded code as posted is that you are starting a new thread every time FixedUpdate is called . That means you're starting 60 new threads per second (by default) which will very quickly cripple your game!

While it's fine to use C# threads in Unity for this sort of work, a better approach would be to create a single thread and allow that to manage the timing, rather than creating a new thread each time the job runs. That would mean creating the thread in Awake() or Start() instead, and then using Thread.Sleep or similar to handle the timing.

Coroutines (as suggested by Mihai in his answer) are great for fixing the timing of events, but they still run on the game thread: if you put your DB code in a coroutine, you'll still see pauses when it runs. If you must run this DB access every second, you need it in a proper thread.

That said, have you considered that the DB access might be unnecessary? A more performant model might be to cache all of the data up front and use it from memory when you need it. (This might not be possible if the data is very dynamic, or if you're running in a memory-restricted environment like a mobile device...)

Answer 2

Whatever you do, you need to stop accessing your database every frame.

You only need the result only once every 60 or frames. You can do this easily by using a variable in which you add up the time passed since last call.

As for multi-threading in Unity, you have three options:

• A multi-threading framework for Unity, like
  https://www.assetstore.unity3d.com/en/#!/content/7285

• C# built-in threading
  You need to be careful not to call Unity specific API from the secondary threads you spawn. It's OK to send over data structures like Vector3, Color, etc., but don't call reference objects like GameObjects or Components.
  https://msdn.microsoft.com/en-us/library/dd321439%28v=vs.110%29.aspx

• Unity Coroutines
  Coroutines are Unity's way of simulating multiple threads. It's quite a powerful tool for getting things to run asynchronously in the same thread
  http://docs.unity3d.com/Manual/Coroutines.html

using System.Threading.Tasks;  

public class Example
{
    void StartOnDifferentThread()
    {
        Task.Factory
            .StartNew(() =>
            {
                FunctionToRun();
            })
            .ContinueWith(task =>
            {
                if (task.IsCompleted)
                {
                    // handle result
                }
                else if (task.IsFaulted)
                {
                    // handle error
                }
            });
    }

    void FunctionToRun()
    {
        // do stuff
    }
}

Answer 3

It finally works now. Just had to add these in retrievefromDB()

public void retrievefromDB(){
      while(true){
                            if (timeStep - prevTimeStep > 99) {
                                    timeStep -= 1; //special for this dataset
   query = "SELECT * FROM GridData2 WHERE timestep=" + timeStep;

                                    if (showParent)
                                            query += " AND (Level != 10)";
                                    else
                                            query += " AND (Level == 10)";

      query += " AND temperature >= " + minTemp + " AND temperature <= " + maxTemp;
                                    dt.Rows.Clear ();
                                    dt = sqlDB.ExecuteQuery (query);

                                    prevTimeStep = timeStep;
                            }
           Thread.Sleep(1);
               }
            }

And put this into the Start() function

testThread = UnityThreadHelper.CreateThread (() =>
                                                     {
            UnityThreadHelper.TaskDistributor.Dispatch (() => retrievefromDB ());

        });

        testThread.Start ();

I'm using the threadhelper from http://forum.unity3d.com/threads/unity-threading-helper.90128/ so u can go and check it out. Thanks to everyone who helped! :)