Repository pattern to get data from dbset.local or database

Question

I am writing a method which will loop over some data and create records in a database, the method makes use of a repository.

Originally it was something like this:

///Lets assume that there is a string array called arr somewhere above this.
var repo = new ItemRepository();

for (int i = 0; i<1000; i++)
{
    if (repo.MyItems.Count(x=>x.ID == arr[i]) == 0)
    {
        //Doesn't Exist Create
        repo.MyItems.Add(new Item(arr[i]));
        repo.Save();
    }
    else
    {
       //Here I will get an instance of the entity that already exists and do some updates to it
    }
}

Which worked but because it was saving to the DB with every iteration it was quite slow, so what I want to do is change it to something like this:

var repo = new ItemRepository();

for (int i = 0; i<1000; i++)
{
    if (repo.MyItems.Count(x=>x.ID == arr[i]) == 0)
    {
        //Doesn't Exist Create
        repo.MyItems.Add(new Item(arr[i]));
    }
    else
    {
       //Here I will get an instance of the entity that already exists and do some updates to it
    }

}

repo.Save();

So that the save is only done AFTER all of the entities have been created, so its one big DB insert rather than thousands.

The problem is that because the items have not been persisted back to my database the repo.MyItems.Count(x=>x.ID == i) doesn't bring anything back (Although the entities have been added to the dbset.Local collection.

I suppose my question is how can I modify my repository to allow me to query the local collection AND the database - because that's the kicker, they may exist in the database OR they may have already been added to the repository and not inserted into the database.

The above is pseudo code, but kind of represents what I am doing, the repository method is something like

public int count(string val)
{
    IQueryable<T> data = _dbSet;

    return data.Count(x=>x.COLUMNNAME == val);
}

Again this is only a representation of what I have in the repo but it should give some idea of what I am going for.

Answer 1

If any one is interested, this is what I implemented and it seemed to work, however I ended up giving up on it as realised that this would pretty much be allowing "Dirty Reads" of the data which had a knock on effect on the rest of the system and we decided that the performance hit of multiple saves was more desirable than reworking big bits of the system - or implementing a repo that allows dirty reads...

Note that GetSingle replaces the Count that I had in the pseudo code above.

public T GetSingle(System.Linq.Expressions.Expression<Func<T, bool>> filter = null)
    {
        //Check the local collection first

        IQueryable<T> localEntities = _dbset.Local.AsQueryable();

        if (filter != null)
        {
            localEntities = localEntities.Where(filter);
        }

        if (localEntities.Count() > 0)
        {
            return localEntities.FirstOrDefault();
        }
        else
        {
            //A local version of this was not found - try the database.

            IQueryable<T> query = this._dbset;

            if (filter != null)
            {
                query = query.Where(filter);
            }

            return query.FirstOrDefault();
        }            
    }

If there is a better way to do this I would really like to know as this is an issue that I am probably going to encounter again and next time I might need to follow through with it!

Answer 2

I hope I understood the problem correctly, this is what I usually do:

        var repo = new ItemRepository();

        Int32 index = 0;
        Int32 localIndex = 0;
        while (index < arr.Length) {
            repo.MyItems.Add(new Item(arr[index]));
            index++;
            localIndex++;
            if (localIndex == 1000) {
                repo.Save();
                localIndex = 0;
            }
        }
        if (localIndex > 0) {
            repo.Save();
        }