Azure Table Storage Query from IronPython

Question

I've got a project that uses IronPython as a scripting engine to perform various tasks. One of those tasks needs to do some table lookup's on the Azure Table storage, however the table layouts are different, and will change often, so I need the model classes to be defined in Python.

Here is the problem I'm running into, whenever I run a query it complains that a base class from my project is not supported by the client library.

Unhandled Exception: System.InvalidOperationException: The type 'IronPython.NewTypes.IPTest.BaseModelClass_1$1' is not supported by the client library.

Python Code:

import clr
import System
clr.AddReference("System.Core")
clr.ImportExtensions(System.Linq)


class MyTable(AzureTableService.BaseModelClass):
    def __new__(self, partitionKey, rowKey):
        self.PartitionKey = partitionKey
        self.RowKey = rowKey
        return super.__new__(self)

    MyTableDetails = "";

#I can manually create an entity, and it recognizes the base class, but not when I try to return from a query
#working = MyTable("10", "10040")
#print working.PartitionKey

y = AzureTableService.GetAzureTableQuery[MyTable]("MyTable")
z = y.Where(lambda c: c.PartitionKey == "10" and c.RowKey == "10040")

print(z.Single())

C# Code:

public class AzureTableService {
    private CloudStorageAccount mStorageAccount;
    public AzureTableService() {
        CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) => {
            var connectionString = ConfigurationManager.AppSettings[configName];
            configSetter(connectionString);
        });
        mStorageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");        
    }

    private TableServiceContext AzureTableServiceContext {
        get {
            var context = mStorageAccount.CreateCloudTableClient().GetDataServiceContext();
            context.IgnoreResourceNotFoundException = true;
            return context;
        }
    }
    public IQueryable<T> GetAzureTableQuery<T>(string TableName) {
        return AzureTableServiceContext.CreateQuery<T>(TableName);
    }

    public class BaseModelClass : TableServiceEntity {
        public BaseModelClass(string partitionKey, string rowKey) : base(partitionKey, rowKey) { }
        public BaseModelClass() : base(Guid.NewGuid().ToString(), String.Empty) { }
    }
}

Is there anything obvious that I'm missing? In my commented code, it seems to recognize my base class properties when I manually create it, however it does not when I try returning it from a query.

Answer 1

The problem you are facing is related to System.Data.Services.Client which is used by Microsoft.WindowsAzure.StorageClient .

It restricts which types can be used in the data services client. This seems to prevent any implementation of IDynamicMetaObjectProvider (basically any dynamic object and therefore any object of an IronPython class) to be used during deserialization of your query result.

I tested the scenario using Azure Storage 1.7.0.0, 2.0.6.0 and 2.1.0.0-rc confirming your results.

You could always have a look at the source and see if another deserializer for AtomPub could be used.