How to perform bulk upload using entity framework on non relational tables

Question

I am having 3 tables

CREATE TABLE [dbo].[TableA](
[TableAID] [bigint] IDENTITY(1,1) NOT NULL,
[PipelineId] [char](10) NULL,
[PipelinePointId] [char](20) NULL,
[Formula] [varchar](500) NULL,
[PipelinePriority] [smallint] NULL

CREATE TABLE [dbo].[TableB](
[TableBID] [smallint] IDENTITY(1,1) NOT NULL,
[UtilityId] [varchar](10) NULL,
[PoolId] [varchar](20) NULL,
[CustomerId] [int] NULL,
[AssetId] [smallint] NULL,
[StartDate] [datetime] NULL,
[EndDate] [datetime] NULL


CREATE TABLE [dbo].[JuntionTable](
[JunctionTableID] [bigint] IDENTITY(1,1) NOT NULL,
[TableAID] [bigint] NOT NULL,
[ParentID] [smallint] NOT NULL, --(ParentID is TableBID)
[AssetType] [nchar](10) NULL,

How can we insert data in this table through entity frame work? we are using Fluent API

Insertion sequence is

1. Insert into TalbeA
2. Insert into TableB
3. Insert into JunctionTable with JuntionTable.TableAID as TalbeAID and JunctionTable.ParentID as TableBID

We cannot have foreign key relation in JunctionTable, since junction table will be holding data from different tables.

Answer 1

You can use SQLBulkCopy to do this, something along the lines of this:

using (var bulkCopy = new SqlBulkCopy(_DbContext.Database.Connection.ConnectionString, SqlBulkCopyOptions.TableLock))
        {
            bulkCopy.BulkCopyTimeout = 1200; // 20 minutes
            //bulkCopy.BatchSize = listTableColumns.Count();
            bulkCopy.BatchSize = 10000;
            bulkCopy.DestinationTableName = "TableA";

            var table = new DataTable();
            var props = TypeDescriptor.GetProperties(typeof(TableA))                                     
                //Dirty hack to make sure we only have system data types                                      
                //i.e. filter out the relationships/collections
                .Cast<PropertyDescriptor>()
                .Where(propertyInfo => propertyInfo.PropertyType.Namespace.Equals("System"))
                .ToArray();
            foreach (var propertyInfo in props) 
            { 
                bulkCopy.ColumnMappings.Add(propertyInfo.Name, propertyInfo.Name); 
                table.Columns.Add(propertyInfo.Name, Nullable.GetUnderlyingType(propertyInfo.PropertyType) ?? propertyInfo.PropertyType); 
            }
            // Add Database ID
            bulkCopy.ColumnMappings.Add("TableAID", "TableAID");
            table.Columns.Add("TableAID", typeof(int));

            var values = new object[props.Length + 1];
            foreach (var item in MyRowsToInsert) 
            { 
                for (var i = 0; i < values.Length - 1; i++) 
                { 
                    values[i] = props[i].GetValue(item);
                }
                table.Rows.Add(values);
            }

            bulkCopy.WriteToServer(table);
        }

Answer 2

Please try the below ,A many-to-many relationship is defined in code by the inclusion of collection properties in each of the entities

public class Book
{
    public int BookId { get; set; }
    public string Title { get; set; }
    public Author Author { get; set; }
    public ICollection<BookCategory> BookCategories { get; set; }
} 

public class Category
{
    public int CategoryId { get; set; }
    public string CategoryName { get; set; }
    public ICollection<BookCategory> BookCategories { get; set; }
}

public class BookCategory
{
    public int BookId { get; set; }
    public Book Book { get; set; }
    public int CategoryId { get; set; }
    public Category Category { get; set; }
}

--you have to configured relationship via the Fluent API for EF Core 1.1.0 to be able to map it successfully as following

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<BookCategory>()
        .HasKey(bc => new { bc.BookId, bc.CategoryId });

    modelBuilder.Entity<BookCategory>()
        .HasOne(bc => bc.Book)
        .WithMany(b => b.BookCategories)
        .HasForeignKey(bc => bc.BookId);

    modelBuilder.Entity<BookCategory>()
        .HasOne(bc => bc.Category)
        .WithMany(c => c.BookCategories)
        .HasForeignKey(bc => bc.CategoryId);
}