Webservice Cache for long running processes Design Ideas

Question

I'm looking for some design ideas

I have a ASP.Net webservice which a website consumes. One of the calls that takes around 13 seconds to retrieve around 70000 rows. 4 seconds on the db and 9 seconds to process on the webserver as there is a processing on every record. This has been optimised as far as I can and has been brought down from an original 42 seconds.

The data doesn't change that often so my idea is to create a cache on the webservice and poll on a timer to update that cache every 30 seconds or so. Then the webservice call retrieves the processed records from the cache

Im looking for design ideas for the best way to approach this. I understand ASP.Net has an input cache dictionary but that wouldn't solve the polling issue so wouldn't I need a singleton anyway, then I have possible threading issues.

Getting pretty confused and not sure if Im on the right lines or if I should be calculating the data and storing it in a DB table, any guidance would be greatly appreciated

UPDATE

As feedback to some of the comments. The website is designed to interact with the ERP Dynamics AX on a customers site, so even though I have some control of the DB layer it is limited (I can add some Select SPs and some indexes but triggers and notifyers of changes is probably a No No)

The latest upgrade for Dynamics AX is in Azure and there is no access to the DB layer, so I will probably have to host the webserver in azure as well. If it is the case and as I need to support all versions it looks like I'm limited to eith Redis or another NoSQL DB is the only option, or I write the result to my own DB table and call from there. Is this definitely the case for Azure?

Answer 1

You can cache your data with Redis and If datas change, you can update cache with sql dependency. Just you need redis and sql dependency I think.

Answer 2

If I were to implement your scenario, I wouldn't prefer polling since there is less point in making repeated calls to the service and keeping service/network busy. Besides if you were to implement a new client, you will have to implement polling again.

Instead use a Dictionary based caching in a static class. You use existing libraries like CacheManager. The general idea is to create a key using the parameters used to make the service call. Then store the results obtained after processing in a ConcurrentDictionary which takes care of access by multiple threads by itself.

Clear the stored result only when the underlying database table (?) have been updated, or if that is too complicated, after every 30 seconds.

Further, you may also implement a similar caching mechanism on your data access layer to bring down the 4 seconds that you currently have. Flush the cached data after the underlying data changes (Add, update, delete, insert operations)!

Answer 3

We implement a polling pattern in ASP.NET that may apply to your use case.

In our Global.ashx , we have:

protected void Application_Start(object sender, EventArgs e)
{
  ConfigurationMonitor.Start();
}

where ConfiguraitonMonitor looks something like this:

public static class ConfigurationMonitor
{
    private static readonly Timer timer = new Timer(PollingInterval);
    public static bool MonitoringEnabled
    {
        get
        {
            return ((timer.Enabled || Working) ? true : false);
        }
    }
    private static int _PollingInterval;
    public static int PollingInterval
    {
        get
        {
            if (_PollingInterval == 0)
            {
                _PollingInterval = (Properties.Settings.Default.ConfigurationPollingIntervalMS > 0) ? Properties.Settings.Default.ConfigurationPollingIntervalMS : 5000;
            }
            return (_PollingInterval);

        }
        set { _PollingInterval = value; }
    }


    private static bool _Working = false;
    public static bool Working
    {
        get { return (_Working); }
    }
    public static void Start()
    {
        Start(PollingInterval);
    }

    /// <summary>
    /// Scans each DLL in a folder, building a list of the ConfigurationMonitor methods to call.
    /// </summary>
    private static List<ConfigurationMonitorAttribute> _MonitorMethods;
    private static List<ConfigurationMonitorAttribute> MonitorMethods
    {
        get
        {
            if (_MonitorMethods == null)
            {
                _MonitorMethods = new List<ConfigurationMonitorAttribute>();
                MonitorMethodsMessage = string.Empty;
                foreach (var assemblyFile in Directory.GetFiles(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "bin"), Properties.Settings.Default.ConfigurtionMonitorDLLPath))
                {
                    var assembly = Assembly.LoadFrom(assemblyFile);
                    foreach (ConfigurationMonitorAttribute monitor in assembly.GetCustomAttributes(typeof(ConfigurationMonitorAttribute), inherit: false))
                    {
                        _MonitorMethods.Add(monitor);
                    }
                }
            }
            return (_MonitorMethods);
        }
    }

    /// <summary>
    /// Resets and instanciates MonitorMethods property to refresh dlls being monitored
    /// </summary>
    public static void LoadMonitoringMethods()
    {
        _MonitorMethods = null;
        List<ConfigurationMonitorAttribute> monitorMethods = MonitorMethods;
    }

    /// <summary>
    /// Initiates a timer to monitor for configuration changes.
    /// This method is invoke on web application startup.
    /// </summary>
    /// <param name="pollingIntervalMS"></param>
    public static void Start(int pollingIntervalMS)
    {
        if (Properties.Settings.Default.ConfigurationMonitoring)
        {
            if (!timer.Enabled)
            {
                LoadMonitoringMethods();
                timer.Interval = pollingIntervalMS;
                timer.Enabled = true;
                timer.Elapsed += new ElapsedEventHandler(OnTimerElapsed);
                timer.Start();
            }
            else
            {
                timer.Interval = pollingIntervalMS;
            }
        }
    }
    public static void Stop()
    {
        if (Properties.Settings.Default.ConfigurationMonitoring)
        {
            if (timer.Enabled)
            {
                timer.Stop();
            }
        }
    }

    /// <summary>
    /// Monitors CE table for changes
    /// </summary>
    /// <param name="sender"></param>
    /// <param name="e"></param>
    private static void OnTimerElapsed(object sender, ElapsedEventArgs e)
    {
        timer.Enabled = false;
        PollForChanges();
        timer.Enabled = true;
    }
    public static DateTime PollForChanges()
    {
        LastPoll = PollForChanges(LastPoll);
        return (LastPoll);
    }
    public static DateTime PollForChanges(DateTime lastPollDate)
    {
        try
        {
            _Working = true;
            foreach (ConfigurationMonitorAttribute monitor in MonitorMethods)
            {
                try
                {
                    lastPollDate = monitor.InvokeMethod(lastPollDate);
                    if (lastPollDate > LastRefreshDate)
                        LastRefreshDate = lastPollDate;
                }
                catch (System.Exception ex)
                {
                    // log the exception; my code omitted for brevity
                }
            }
        }
        catch (System.Exception ex)
        {
            // log the exception; my code omitted for brevity

        }
        finally
        {
            _Working = false;
        }
        return (lastPollDate);
    }

    #region Events
    /// <summary>
    /// Event raised when an AppDomain reset should occur
    /// </summary>
    public static event AppDomainChangeEvent AppDomainChanged;
    public static void OnAppDomainChanged(string configFile, IDictionary<string, object> properties)
    {
        if (AppDomainChanged != null) AppDomainChanged(null, new AppDomainArgs(configFile, properties));
    }
    #endregion
}

When we have a use case that wants to 'participate' in this polling mechanism, we tag some method with an attribute:

[assembly: ConfigurationMonitorAttribute(typeof(bar), "Monitor")]
namespace foo
{
  public class bar 
  {
    public static DateTime Monitor(DateTime lastPoll)
    {
      // do your expensive work here, setting values in your cache
    }
  }
}

Our pattern of having the method triggered by ConfigurationMonitor returning a DateTime was a fairly bizarre edge case. You could certainly go with a void method.

where the ConfigurationMonitorAttribute is something like this:

[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public class ConfigurationMonitorAttribute : Attribute
{
    private Type _type;
    private string _methodName;

    public ConfigurationMonitorAttribute(Type type, string methodName)
    {
        _type = type;
        _methodName = methodName;
    }

    public Type Type
    {
        get
        {
            return _type;
        }
    }

    public string MethodName
    {
        get
        {
            return _methodName;
        }
    }

    private MethodInfo _Method;
    protected MethodInfo Method
    {
        get
        {
            if (_Method == null)
            {
                _Method = Type.GetMethod(MethodName, BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
                if (_Method == null)
                    throw new ArgumentException(string.Format("The type {0} doesn't have a static method named {1}.", Type, MethodName));

            }
            return _Method;
        }
    }
    public DateTime InvokeMethod(DateTime lastPoll)
    {
        try
        {
            return (DateTime)Method.Invoke(null, new object[] { lastPoll });
        }
        catch (System.Exception err)
        {
            new qbo.Exception.ThirdPartyException(string.Format("Attempting to monitor {0}/{1} raised an error.", _type, _methodName), err);
        }
        return lastPoll;
    }
}

Answer 4

There are a few other things to think about in this scenario other than just saying "I want to add caching".

1. If you are running in Azure or in a web-farm you need a centralized cache (REDIS or the likes) as a Memory Cache will be destroyed and re-created with your site, and local to one server in the farm, so you won't necessarily see the performance gains.

2. If you do setup a REDIS Cache make sure you take care in configuring it. The coding must be just so to account for connections, and if you don't do it right, you will end up overrunning your connection pool.

3. This is more just on your situation, but even 4 seconds to return 70k records seem high. Have you run through an execution plan to see if there are missing indexes or optimizations with CTEs that can be applied?

Answer 5

You can set expiration policies for a cache and load on demand. You can also have more then one level of caching, one as a distributed and one local as the local version will always be fastest. I prefer loading on demand vs polling as with polling you are always refreshing the data, even at times when no one is listening. If you do go multi tier you can poll for your distributed cache, and load the local cache on demand. That's about as efficient as you'll get.