通过 ASP.NET Core REST API 并发请求后来自 MongoDb 的响应缓慢

Question

I'm running a web service based on ASP.NET Core 2.2. External clients call the REST API in regular intervalls. So it is possible, that the API is hit with about 50 nearly concurrent requests (every 10 to 40 min.).

Problem: Some minutes after the concurrent requests the service's response time is high. The response time increased from < 1 s (for single requests) to 40 sec.

Remarks:

• The query retrieves items (< 100 in total) from about three different collections (filtering by a single property included in the index, some by id; holding < 5000 records).
• For persistence the MongoDB.Driver 2.9.1 and a single installation of MongoDB 4.0.12 are used.
• Updating to the latest driver and MongoDB 4.0 has not improved the situation.
• The MongoDB database profiler shows no slow queries.
• The output from mongostat shows about 90 open connections.
• The tests have been executed on an older machine (some Haswell Core i7, 32 GB RAM) running Windows 10 (db and application on same machine).
• The MongoDB installation runs with default configuration.
• Continouus load tests (eg 3000 x 50 concurrent requests) have shown, that after the initial ramp-up phase (with slow requests) the response time drops to < 5 sec, what I would consider as normal.
  • There are some write operations on those collections (only from one client eg 10 per hour).

Questions:

• Any recommendations for performance tuning?
• Any performance indicators to check?
• Issue with the connection pooling?

EDIT: Using the ClusterConfigurator of the MongoClientSettings available from the MongoDB Driver package for C# I measured an increase of duration of a query up to 00:00:04.5022352 seconds (from 00:00:00.0005432).

I noticed that opLatencies reads are increasing during 50 nearly concurrent requests: opLatencies.reads.latency: from 68723 to 202886, opLatencies.reads.ops: from 449 to 1053

Background:

The MongoClient is created in following class:

public class MongoDbContext : IContext {
    private readonly MongoUrl _url;
    private MongoClient _client;
    private IMongoDatabase _db;

    public MongoDbContext (string connectionString) {
        _url = MongoUrl.Create (connectionString);
    }

    public IMongoDatabase GetDatabase () {
        return _db ?? (_db = GetClient ().GetDatabase (_url.DatabaseName));
    }

    public MongoClient GetClient () {
        return _client ?? (_client = new MongoClient (_url));
    }
}

The context is injected in the Startup.cs and used in serveral repositories.

//... Startup.cs
var context = new MongoDbContext(connectionString);    
services.AddSingleton<IContext>(context);
// Other services and repositories ...
services.AddTransient<IWeatherRepository, WeatherRepository>();

The repositories use following base class:

public abstract class MongoDbRepository<T> : IRepository<T> where T : Entity<long> {

    private IMongoCollection<T> _collection { get; set; }

    protected IContext Context { get; set; }

    protected abstract IMongoCollection<T> GetCollection ();

    protected IMongoCollection<T> Collection {
        get { return _collection ?? (_collection = GetCollection ()); }
        set { _collection = value; }
    }

    public MongoDbRepository (IContext context) {     
        Context = context;
    }

    /// ...

    public async Task<IEnumerable<T>> Find (Expression<Func<T, bool>> filter) {
        return await Collection.Find (filter).ToListAsync ();
    }
}

Answer 1

你有没有试过对这些操作使用异步方法可能会帮助你改善并发操作响应。（在数据库上下文中）

Answer 2

I would not recommend to use Singleton for DB Context. This object should be released as soon as it is not needed. Use Scoped instead, it will be shared among your repositories within request and then will be released:

services.AddScoped<IContext, MongoDbContext>(provider => new MongoDbContext(connectionString));