Avoiding duplicates when inserting multiple data into MongoDB

Question

Hi I want to insert multiple data and avoid duplicates in my programme. I have implemented the code to insert multiple data to mongoDB using asp.net core web api and it's working fine. I tried so many things but still I couldn't find a way to avoid duplicate records when inserting multiple records. Can you guys help me please:)

I want to avoid inserting duplicates by using employeeid.

this is my controller

using HelloApi.Models;
using HelloApi.Services;
using Microsoft.AspNetCore.Mvc;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace HelloApi.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    [Produces("application/json")]
    public class EmployeeController : ControllerBase
    {
        private readonly EmployeeService _employeeService;

        public EmployeeController(EmployeeService employeeService)
        {
            _employeeService = employeeService;
        }

        //.....................................Create..............................................

        public Task Create(IEnumerable<Employee> employees)
        {
            return _employeeService.Create(employees);

        }
    }
}

This is my model class

using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using System.ComponentModel.DataAnnotations;

namespace HelloApi.Models
{
    public class Employee
    {
        [BsonId]
        [BsonRepresentation(BsonType.ObjectId)]
        [Key]
        public string Id { get; set; }

        [Required]
        [BsonElement("employeeid")]
        public string employeeid { get; set; }

        [Required]
        [BsonElement("firstname")]
        public string firstname { get; set; }

        [Required]
        [BsonElement("lastname")]
        public string lastname { get; set; }

        [Required]
        [BsonElement("age")]
        public int age { get; set; }

        [Required]
        [BsonElement("address")]
        public string address { get; set; }

        [Required]
        [BsonElement("telephone")]
        public int telephone { get; set; }

    }
}

This is my service class

using HelloApi.Models;
using MongoDB.Driver;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace HelloApi.Services
{
    public class EmployeeService
    {
        private readonly IMongoCollection<Employee> _employee;

        public EmployeeService(IHelloApiDatabaseSettings settings)
        {
            var client = new MongoClient(settings.ConnectionString);
            var database = client.GetDatabase(settings.DatabaseName);

            _employee = database.GetCollection<Employee>(settings.employeeCollectionName);
        }

        //.....................................Create..............................................

        public Task Create(IEnumerable<Employee> employees)
        {
            return _employee.InsertManyAsync(employees);
        }
    }
}

This is the POST request in Postman and it is working fine but submit duplicates.

Really appreciate if you guys can help me with this:)

Answer 1

It seems you need to create unique index on employeeId field to avoid duplicates:

 db.employee.createIndex( { "employeeId": 1 }, { unique: true } )

Answer 2

I think you need to add an extra step to declare your Id field as a Primary Key. [Key] should only work for RDBMS. For MongoDB i found another post here to define a field as a unique key.

(Actually MongoDB doesn't really allow for a Primary Key besides the auto-generated '_id' field, afaik, but you can create unique keys. )

Try looking through the answers posted here: How to set a primary key in MongoDB?

Answer 3

This Might be helpful. Use below InsertORUpdateEmployee function to find and update using employeeid.

public Task<string> Create(IEnumerable<Employee> employees)
{
    return Task.Run(() =>
    {
        return InsertORUpdateEmployee(employees);
    });
}

//Insert or Update Employees
private string InsertORUpdateEmployee(IEnumerable<Employee> employees)
{
    try
    {
        foreach (Employee emp in employees)
        {
            var empId = emp.employeeid;
            var DB = Client.GetDatabase("Employee");
            var collection = DB.GetCollection<Employee>("EmployeeDetails");
            
            //Find Employee using employeeid
            var filter_id = Builders<Employee>.Filter.Eq("employeeid", empId);
            var entity = collection.Find(filter_id).FirstOrDefault();
            //Insert
            if (entity == null)
            {
                collection.InsertOne(emp);
            }
            else
            {
                //Update
                var update = collection.FindOneAndUpdateAsync(filter_id, Builders<Employee>.Update
                    .Set("firstname", emp.firstname)
                    .Set("lastname", emp.lastname)
                    .Set("age", emp.age)
                    .Set("address", emp.address)
                    .Set("telephone", emp.telephone));
            }
        }
        return "Insert or Updated Succesfully";
    }

    catch (Exception ex)
    {
        return ex.ToString();
    }

}