向 API 发布多个请求，但不重复使用连接

Question

I have a console application which is calling a third party API, basically it takes records from a database and pushes them to the third party via their WCF API.

For various reasons (Mainly the third party API being very slow - 7 seconds to respond) we want to post multiple records in parallel so we have started doing this, however we are now seeing some strange behaviour from the third party API where it is duplicating records.

It has been suggested to us by the developers of the API that this is because we are sending the requests over the same connection (which makes sense as .net will reuse connections) and they dont/cant/wont support that, they will only support one request over one connection and then the connection must be closed.

My question is, How do I do this in .net core (2.2) We are currently using a HttpClient which I'd expect to reuse connections where possible - how can I guarantee that we use a new connection for each request?

Answer 1

I have after some digging worked out what the problem is and there is no way to fix it.

The process is :

1. We Post into the API
2. The API creates a record in a staging table in the database with a flag showing a status of "to be processed" and then starts polling the table waiting for change.
3. The API then invokes an executable on the server which looks at the staging table for any records with a status of "to be processed"
4. The executable does its processing and then changes the status on the record to "complete"
5. The API which has been polling the table sees the changed status, reads the record and returns the result to the client.

All fine if you only ever post one record at a time. But as Im executing in parallel what is happening is :

1. We call the API 10 times within a few ms of each other
2. The API creates 10 records in the staging table all with the "to be processed" status.
3. The API starts polling the staging table for changes and at the same time invokes the executable TEN TIMES
4. All 10 instances of the executable read all 10 records and process them - each instance unaware that there are another 9 instances all doing the same thing.
5. All 10 instances of the executable finish processing and change the status on the staging table to "complete"
6. The API sees the status change and returns all of the changed records to me in the response - So each of the 10 requests I sent gets 10 records returned to it.

Needless to say, we have entered into discussions with the provider of the API, it might be EOL and pretty much out of support but we're paying for licencing of this thing and this is a really stupid process that they need to provide a fix or a workaround for.

So in the end it was nothing to do with reusing connections, dont know why we were told it was.