Lazy Loading for Owned Types

Question

I am moving my first steps towards Domain Driven Design using Entity Framework Core. I have a User entity that, in a simplified version, has only Id and ProfilePhoto . However, I want to store profile photos in a different table, that is why I created an Owned Type containing the profile photo and configured in this way:

User:

public class User
{
    private int id;
    public int Id => this.id;

    //private UserProfilePhoto userProfilePhoto;
    public virtual UserProfilePhoto UserProfilePhoto { get; set; }

    private User()
    {
    }

    public static User Create(byte[] profilePhoto)
    {
        var user = new User();
        user.UserProfilePhoto = new UserProfilePhoto(profilePhoto);

        return user;
    }

    public void SetProfilePhoto(byte[] profilePhoto)
    {
        this.UserProfilePhoto = new UserProfilePhoto(profilePhoto);
    }
}

UserProfilePhoto:

public class UserProfilePhoto
{
    public byte[] ProfilePhoto { get; private set; }

    public UserProfilePhoto(byte[] profilePhoto)
    {
        this.ProfilePhoto = profilePhoto;
    }
}

DbContext configuration:

public class ModelContext : DbContext
{
    public ModelContext(DbContextOptions<ModelContext> options) : base(options)
    {
    }

    public DbSet<User> Users { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        base.OnModelCreating(modelBuilder);

        OnUserModelCreating(modelBuilder);
    }

    protected void OnUserModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<User>()
            .HasKey(u => u.Id);

        modelBuilder.Entity<User>()
            .Property(u => u.Id)
            .HasField("id");

        modelBuilder.Entity<User>()
            .OwnsOne(u => u.UserProfilePhoto, builder =>
            {
                builder.ToTable("UserProfilePhoto");

                builder.Property(u => u.ProfilePhoto)
                    .IsRequired();
            });
    }
}

I chose to use an Owned type because I want the profile photo to be accessible only from the user entity. with a one-to-one mapping, I could still access the UserProfilePhoto table using context.Set<UserProfilePhoto>() for example and, for what I read about DDD aggregates, this could mean skipping User business logic. So, I migrated and the database model is just like I expected it to be: the UserProfilePhoto table with a primary and foreign key to User.Id .

Obviously in my queries I do not want to load the entire User entity every time, so I enabled Lazy Loading, unsuccessfully. This is the code I tried in a unit test:

protected ModelContext GetModelContext(DbContextOptionsBuilder<ModelContext> builder)
{
    builder
        .UseLoggerFactory(loggerFactory)
        .UseLazyLoadingProxies()
        .EnableDetailedErrors();

    var ctx = new ModelContext(builder.Options);
    ctx.Database.EnsureCreated();

    return ctx;
}

[TestMethod]
public async Task TestMethod1()
{
    var builder = new DbContextOptionsBuilder<ModelContext>()
        .UseSqlServer(...);
    var ctx = this.GetModelContext(builder);
    var user = User.Create(new byte[] { });

    try
    {
        await ctx.Users.AddAsync(user);
        await ctx.SaveChangesAsync();

        var users = ctx.Users;

        foreach (var u in users)
        {
            Console.WriteLine(u.Id);
        }
    }
    finally
    {
        ctx.Users.Remove(user);
        await ctx.SaveChangesAsync();
        ctx.Database.EnsureDeleted();
    }
}

And here the SQL generated:

SELECT [u].[Id], [u0].[UserId], [u0].[ProfilePhoto]
FROM [Users] AS [u]
LEFT JOIN [UserProfilePhoto] AS [u0] ON [u].[Id] = [u0].[UserId]

I do not exactly know if it works, but injecting an ILazyLoader is not an solution for me, on the other hand, it feels like dirtying the model.

My doubt is that Owned types do not bind to the principal entity through actual navigation properties, so creating proxy for them is not supported.

What is wrong with my approach? Is it DDD? And if so, how can I lazily load owned entities?

I found an issue on Github related to this, although it does not answer my question.

---

Edit

my goal is to prevent the access to the UserProfilePhoto table from EF api (See comments). If I managed to do this, then protecting my UserProfilePhoto class and encapsulating it in the User class would be easy, something like this:

User

...
protected virtual UserProfilePhoto UserProfilePhoto { get; set; }

public void SetProfilePhoto(byte[] profilePhoto)
{
    this.UserProfilePhoto.SetProfilePhoto(profilePhoto);
}

public byte[] GetProfilePhoto()
{
    return this.UserProfilePhoto.ProfilePhoto;
}
...

I tried this code with a one-to-one mapping and works perfectly, even with lazy loading. How could I do this with only Owned Types? are there other ways?

Answer 1

EF Core loads owned types automatically when the owner gets loaded (from Owned Entity Types: Querying owned types )

When querying the owner the owned types will be included by default. It is not necessary to use the Include method, even if the owned types are stored in a separate table.

Therefore using owned types does not fulfill your requirement of being loaded only on demand.

(You can tinker with Metadata.PrincipalToDependent.SetIsEagerLoaded(false) etc., but this is very much unsupported, unlikely to work in all cases and could break any time.)

Options without using owned types (in order of recommendation)

• Override DbContext.Set<>() , DbContext.Find() etc. and throw if called inappropriately
• Implement a traditional custom Unit-of-Work and Repository pattern, that gives you full control over the API exposed (trades flexibility for control)
• Add an expression visitor early to the query pipeline (register IQueryTranslationPreprocessorFactory and derive from RelationalQueryTranslationPreprocessorFactory ), that throws if a DbSet<UserProfilePhoto> is used anywhere in the query
• Provide your own IDbSetSource (and InternalDbSet ) implementation (both internal) and throw if called inappropriately

Overriding DbContext methods

Generally, just overriding DbContext.Set<>() , DbContext.Find() etc. should be the simplest solution. You could decorate the types that you don't want to be queried directly with a custom attribute and then simply just check, that TEntity etc. has not have been decorated with this custom attribute.

For easier maintainability, all the overridden methods can be moved to a base class, that can also perform some runtime check to ensure, that all methods in question have been overridden (of course those checks could also be done by a unit test).

Here is a sample demonstrating this approach:

using System;
using System.Diagnostics;
using System.Linq;
using System.Reflection;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;

namespace IssueConsoleTemplate
{
    [AttributeUsage(AttributeTargets.Class)]
    public sealed class DontRootQueryMeAttribute : Attribute
    {
    }
    
    public class User
    {
        public int Id { get; private set; }

        public virtual UserProfilePhoto UserProfilePhoto { get; set; }

        public static User Create(byte[] profilePhoto)
        {
            var user = new User
            {
                UserProfilePhoto = new UserProfilePhoto(profilePhoto)
            };

            return user;
        }
    }

    [DontRootQueryMeAttribute]
    public class UserProfilePhoto
    {
        public int Id { get; set; }
        public byte[] ProfilePhoto { get; private set; }

        public UserProfilePhoto(byte[] profilePhoto)
        {
            ProfilePhoto = profilePhoto;
        }
    }
    
    public abstract class ModelContextBase : DbContext
    {
        private static readonly string[] OverriddenMethodNames =
        {
            nameof(DbContext.Set),
            nameof(DbContext.Query),
            nameof(DbContext.Find),
            nameof(DbContext.FindAsync),
        };

        static ModelContextBase()
        {
            var type = typeof(ModelContextBase);

            var overriddenMethods = type
                .GetRuntimeMethods()
                .Where(
                    m => m.IsPublic &&
                         !m.IsStatic &&
                         OverriddenMethodNames.Contains(m.Name) &&
                         m.GetRuntimeBaseDefinition() != null)
                .Select(m => m.GetRuntimeBaseDefinition())
                .ToArray();

            var missingOverrides = type.BaseType
                .GetRuntimeMethods()
                .Where(
                    m => m.IsPublic &&
                         !m.IsStatic &&
                         OverriddenMethodNames.Contains(m.Name) &&
                         !overriddenMethods.Contains(m))
                .ToArray();

            if (missingOverrides.Length > 0)
            {
                throw new InvalidOperationException(
                    $"The '{nameof(ModelContextBase)}' class is missing overrides for {string.Join(", ", missingOverrides.Select(m => m.Name))}.");
            }
        }
        
        private void EnsureRootQueryAllowed<TEntity>()
            => EnsureRootQueryAllowed(typeof(TEntity));

        private void EnsureRootQueryAllowed(Type type)
        {
            var rootQueriesAllowed = type.GetCustomAttribute(typeof(DontRootQueryMeAttribute)) == null;
            
            if (!rootQueriesAllowed)
                throw new InvalidOperationException($"Directly querying for '{type.Name}' is prohibited.");
        }

        public override DbSet<TEntity> Set<TEntity>()
        {
            EnsureRootQueryAllowed<TEntity>();
            return base.Set<TEntity>();
        }

        public override DbQuery<TQuery> Query<TQuery>()
        {
            EnsureRootQueryAllowed<TQuery>();
            return base.Query<TQuery>();
        }

        public override object Find(Type entityType, params object[] keyValues)
        {
            EnsureRootQueryAllowed(entityType);
            return base.Find(entityType, keyValues);
        }

        public override ValueTask<object> FindAsync(Type entityType, params object[] keyValues)
        {
            EnsureRootQueryAllowed(entityType);
            return base.FindAsync(entityType, keyValues);
        }

        public override ValueTask<object> FindAsync(Type entityType, object[] keyValues, CancellationToken cancellationToken)
        {
            EnsureRootQueryAllowed(entityType);
            return base.FindAsync(entityType, keyValues, cancellationToken);
        }

        public override TEntity Find<TEntity>(params object[] keyValues)
        {
            EnsureRootQueryAllowed<TEntity>();
            return base.Find<TEntity>(keyValues);
        }

        public override ValueTask<TEntity> FindAsync<TEntity>(params object[] keyValues)
        {
            EnsureRootQueryAllowed<TEntity>();
            return base.FindAsync<TEntity>(keyValues);
        }

        public override ValueTask<TEntity> FindAsync<TEntity>(object[] keyValues, CancellationToken cancellationToken)
        {
            EnsureRootQueryAllowed<TEntity>();
            return base.FindAsync<TEntity>(keyValues, cancellationToken);
        }

        // Add other overrides as needed...
    }
    
    public class ModelContext : ModelContextBase
    {
        public DbSet<User> Users { get; set; }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder
                .UseSqlServer(
                    @"Data Source=.\MSSQL14;Integrated Security=SSPI;Initial Catalog=So63887500_01")
                .UseLoggerFactory(LoggerFactory.Create(b => b
                    .AddConsole()
                    .AddFilter(level => level >= LogLevel.Information)))
                .EnableSensitiveDataLogging()
                .EnableDetailedErrors();
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder);

            OnUserModelCreating(modelBuilder);
        }

        protected void OnUserModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<User>(
                entity =>
                {
                    entity.HasOne(e => e.UserProfilePhoto)
                        .WithOne()
                        .HasForeignKey<UserProfilePhoto>(e => e.Id);
                });
        }
    }

    internal static class Program
    {
        private static void Main()
        {
            var accessingSetThrows = false;

            using (var ctx = new ModelContext())
            {
                ctx.Database.EnsureDeleted();
                ctx.Database.EnsureCreated();

                var user = User.Create(new byte[] { });

                ctx.Users.Add(user);
                ctx.SaveChanges();
                
                // Make sure, that UserProfilePhoto cannot be queried directly.
                try
                {
                    ctx.Set<UserProfilePhoto>()
                        .ToList();
                }
                catch (InvalidOperationException)
                {
                    accessingSetThrows = true;
                }
                
                Debug.Assert(accessingSetThrows);
            }

            // No eager loading by default with owned type here.
            using (var ctx = new ModelContext())
            {
                var users = ctx.Users.ToList();

                Debug.Assert(users.Count == 1);
                Debug.Assert(users[0].UserProfilePhoto == null);
            }

            // Explicitly load profile photo.
            using (var ctx = new ModelContext())
            {
                var users = ctx.Users.ToList();
                ctx.Entry(users[0]).Reference(u => u.UserProfilePhoto).Load();
                
                Debug.Assert(users.Count == 1);
                Debug.Assert(users[0].UserProfilePhoto != null);
            }
        }
    }
}

Providing an IQueryTranslationPreprocessorFactory implementation

An expression visitor can be used to solve the issue by using an IQueryTranslationPreprocessorFactory implementation to search the query for a specific expression, that is only added when the new InternalQuery() extension method is called and throwing, if it is missing and a non-root entity is being queried. In practice, this should be good enough to make sure, that nobody in the team queries non-root objects by accident.

(You could also add an internal class instance as a constant parameter to the method call expression, that is then evaluated later in the expression visitor to ensure, that the caller really had internal access to the InternalQuery() methods. But this is just icing on the cake and unnecessary in practice, since developers could use reflection to bypass any access restrictions anyway. So I wouldn't bother to implement this.)

Here it the implementation (using a custom interface instead of a custom attribute to mark entities that should not be queried directly):

using System;
using System.Diagnostics;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.ChangeTracking;
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

namespace IssueConsoleTemplate
{
    #region Models
    
    public class User
    {
        public int Id { get; private set; }

        public virtual UserProfilePhoto UserProfilePhoto { get; set; }

        public static User Create(byte[] profilePhoto)
        {
            var user = new User
            {
                UserProfilePhoto = new UserProfilePhoto(profilePhoto)
            };

            return user;
        }
    }
    
    public class UserProfilePhoto : INonRootQueryable
    {
        public int Id { get; set; }
        public byte[] ProfilePhoto { get; private set; }

        public UserProfilePhoto(byte[] profilePhoto)
        {
            ProfilePhoto = profilePhoto;
        }
    }

    #endregion

    #region Custom implementations

    public interface INonRootQueryable
    {
    }
    
    public class CustomQueryTranslationPreprocessorFactory : IQueryTranslationPreprocessorFactory
    {
        private readonly QueryTranslationPreprocessorDependencies _dependencies;
        private readonly RelationalQueryTranslationPreprocessorDependencies _relationalDependencies;

        public CustomQueryTranslationPreprocessorFactory(
            QueryTranslationPreprocessorDependencies dependencies,
            RelationalQueryTranslationPreprocessorDependencies relationalDependencies)
        {
            _dependencies = dependencies;
            _relationalDependencies = relationalDependencies;
        }

        public virtual QueryTranslationPreprocessor Create(QueryCompilationContext queryCompilationContext)
            => new CustomQueryTranslationPreprocessor(
                _dependencies,
                _relationalDependencies,
                queryCompilationContext);
    }

    public class CustomQueryTranslationPreprocessor : RelationalQueryTranslationPreprocessor
    {
        public CustomQueryTranslationPreprocessor(
            QueryTranslationPreprocessorDependencies dependencies,
            RelationalQueryTranslationPreprocessorDependencies relationalDependencies,
            QueryCompilationContext queryCompilationContext)
            : base(dependencies, relationalDependencies, queryCompilationContext)
        {
        }

        public override Expression Process(Expression query)
        {
            query = new ThrowOnNoneRootQueryableViolationExpressionVisitor().Visit(query);
            return base.Process(query);
        }
    }
    
    public class ThrowOnNoneRootQueryableViolationExpressionVisitor : ExpressionVisitor
    {
        private bool _isInternalQuery;

        protected override Expression VisitMethodCall(MethodCallExpression node)
        {
            if (node.Method.GetGenericMethodDefinition() == CustomQueryableExtensions.InternalQueryMethodInfo)
            {
                _isInternalQuery = true;
                return node.Arguments[0];
            }
            
            return base.VisitMethodCall(node);
        }

        protected override Expression VisitConstant(ConstantExpression node)
        {
            var expression = base.VisitConstant(node);
            
            // Throws if SomeEntity in a DbSet<SomeEntity> implements INonRootQueryable and the query was not chained
            // to the `InternalQuery()` extension method.
            return !_isInternalQuery &&
                   node.Type.IsGenericType &&
                   node.Type.GetGenericTypeDefinition() == typeof(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable<>) &&
                   node.Type.GenericTypeArguments.Length == 1 &&
                   typeof(INonRootQueryable).IsAssignableFrom(node.Type.GenericTypeArguments[0])
                ? throw new InvalidOperationException($"Directly querying for '{node.Type.Name}' is prohibited.")
                : expression;
        }
    }
    
    internal static class CustomQueryableExtensions
    {
        internal static readonly MethodInfo InternalQueryMethodInfo
            = typeof(CustomQueryableExtensions)
                .GetTypeInfo()
                .GetDeclaredMethods(nameof(InternalQuery))
                .Single(m => m.GetParameters().Length == 1 &&
                             m.GetParameters()[0].ParameterType.Namespace == $"{nameof(System)}.{nameof(System.Linq)}" &&
                             m.GetParameters()[0].ParameterType.Name.StartsWith(nameof(IQueryable)) &&
                             m.GetParameters()[0].ParameterType.GenericTypeArguments.Length == 1);

        internal static IQueryable<TSource> InternalQuery<TSource>(this IQueryable<TSource> source)
            => source.Provider.CreateQuery<TSource>(
                Expression.Call(
                    null,
                    InternalQueryMethodInfo.MakeGenericMethod(typeof(TSource)),
                    source.Expression));

        internal static IQueryable<TProperty> InternalQuery<TEntity, TProperty>(this ReferenceEntry<TEntity, TProperty> source)
            where TEntity : class
            where TProperty : class
            => source.Query()
                .InternalQuery();
    }
    
    #endregion
    
    public class ModelContext : DbContext
    {
        public DbSet<User> Users { get; set; }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            // Register the custom type IQueryTranslationPreprocessorFactory.
            // Since this is a console program, we need to create our own ServiceCollection
            // for this.
            // In an ASP.NET Core application, the AddSingleton call can just be added to
            // the general service configuration method.
            var serviceProvider = new ServiceCollection()
                .AddEntityFrameworkSqlServer()
                .AddSingleton<IQueryTranslationPreprocessorFactory, CustomQueryTranslationPreprocessorFactory>()
                .AddScoped(
                    s => LoggerFactory.Create(
                        b => b
                            .AddConsole()
                            .AddFilter(level => level >= LogLevel.Information)))
                .BuildServiceProvider();

            optionsBuilder
                .UseInternalServiceProvider(serviceProvider) // <-- use our ServiceProvider
                .UseSqlServer(@"Data Source=.\MSSQL14;Integrated Security=SSPI;Initial Catalog=So63887500_05")
                .EnableSensitiveDataLogging()
                .EnableDetailedErrors();
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder);
            OnUserModelCreating(modelBuilder);
        }

        protected void OnUserModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<User>(
                entity =>
                {
                    entity.HasOne(e => e.UserProfilePhoto)
                        .WithOne()
                        .HasForeignKey<UserProfilePhoto>(e => e.Id);
                });
        }
    }

    internal static class Program
    {
        private static void Main()
        {
            var accessingSetThrows = false;

            using (var ctx = new ModelContext())
            {
                ctx.Database.EnsureDeleted();
                ctx.Database.EnsureCreated();

                var user = User.Create(new byte[] { });

                ctx.Users.Add(user);
                ctx.SaveChanges();
            }

            // Make sure, that UserProfilePhoto cannot be queried directly by default.
            using (var ctx = new ModelContext())
            {
                try
                {
                    ctx.Set<UserProfilePhoto>()
                        .ToList();
                }
                catch (InvalidOperationException)
                {
                    accessingSetThrows = true;
                }
                
                Debug.Assert(accessingSetThrows);
            }

            // Make sure, that UserProfilePhoto can be queried directly, when using the `InternalQuery()` extension
            // method.
            using (var ctx = new ModelContext())
            {
                var userProfilePhotos = ctx.Set<UserProfilePhoto>()
                    .InternalQuery()
                    .ToList();
                
                Debug.Assert(userProfilePhotos.Count == 1);
            }

            // No eager loading of referenced types by default.
            using (var ctx = new ModelContext())
            {
                var users = ctx.Users.ToList();

                Debug.Assert(users.Count == 1);
                Debug.Assert(users[0].UserProfilePhoto == null);
            }

            // Eager loading of referenced types is allowed, when using the `InternalQuery()` extension method.
            using (var ctx = new ModelContext())
            {
                var users = ctx.Users
                    .Include(u => u.UserProfilePhoto)
                    .InternalQuery()
                    .ToList();

                Debug.Assert(users.Count == 1);
                Debug.Assert(users[0].UserProfilePhoto != null);
            }

            // Explicitly load profile photo, when using the `InternalQuery()` extension method.
            using (var ctx = new ModelContext())
            {
                var users = ctx.Users.ToList();
                ctx.Entry(users[0])
                    .Reference(u => u.UserProfilePhoto)
                    .InternalQuery()
                    .Load();

                Debug.Assert(users.Count == 1);
                Debug.Assert(users[0].UserProfilePhoto != null);
            }
        }
    }
}

Answer 2

Should bytes really be part of the domain? Do you actually run any business logic on those bytes in the user profile context? Is there really a use case where you'd want to access the bytes from within the User AR?

If not then perhaps it makes more sense decoupling the bytes storage from the photo's metadata and introduce a ProfilePhoto VO with a storageUrl/storageId property to locate the bytes.

Don't forget that your domain model should be designed for commands, not queries & the presentation layer.

Granted, now you can't easily have ACID properties when storing the bytes & the AR's data in the DB, but it's usually easy to cope with that with a cleanup process.

If you don't need profile photo's metadata in User to enforce business rules then you may also consider making ProfilePhoto it's own AR.

Finally, I think trying to prevent ORM misuse is unnecessary. The ORM should be seen as a low-level API which shouldn't ever be used directly to change AR states. I think it's safe to assume developers will have enough rigour to respect that rule just like they should respect the overall system's architecture. If they don't you have bigger problems. If it was as easy as adding a private modifier to a member then sure, but it seems to be needing a lot of efforts so I'd just go the pragmatic way...

Answer 3

I found a temporary solution:

modelBuilder.Entity<User>()
    .OwnsOne(u => u.UserProfilePhoto, builder =>
    {
        builder.Metadata.IsOwnership = false;
        builder.Metadata.IsRequired = false;
        builder.Metadata.PrincipalToDependent.SetIsEagerLoaded(false);

        builder.ToTable("UserProfilePhoto");

        builder.Property(u => u.ProfilePhoto)
            .IsRequired();
    });

I do not like it and I guess EF allows you to configure that in other, more clear, ways. I am not accepting this answer, hoping someone else could point me in the right direction.

---

EDIT: proxy works this way but when a User is deleted, the association with the UserProfilePhoto is severed:

The association between entities ' User ' and ' UserProfilePhoto ' with the key value '{UserId: 1}' has been severed but the relationship is either marked as 'Required' or is implicitly required because the foreign key is not nullable. If the dependent/child entity should be deleted when a required relationship is severed, then setup the relationship to use cascade deletes.

I even tried to specify through metadata the DeleteBehaviour.Cascade option but it probably breaks an internal constraint.

Moreover, it is now accessible via DbContext.Set<UserProfilephoto>() , which is not what I want.