Slow performance when querying remote db - is this normal?

Question

I am doing some loading from an oracle db, using ODP.NET.

In its current implementation the code does something along the lines of:

query entityIds to load based on criteria  
    foreach entityId
        load attributes 
        query geometries that exist
        foreach geometry that exists
            load geometry
        next
    next

when the DB is on the local network criteria which load 133 entities takes a couple of seconds to load all 133 entities.

When the db is a remote db hosted on a VM in a data center on the other side of the world this takes about 3.5 mins to load them all.

The particularly slow bit seems to be the querying the geometry. In initial testing (in TOAD - not in the service loading code) it seems to take about 2 secs to load the geometry for a single entity using the remote machine. If we change the query to load all the geometries in a single go, it still seems to take 2 secs. This sort of implies that it isn't the network overhead (as the amount of data being returned is much more for the query which returns all the geometries, but the time is the same).

Is this sort of performance overhead for a remote db vs local expected? Why does doing each query separately take so much longer than doing them all in one go? Is there anything we can do to mitigate this (apart from do all the queries in one go)?

Answer 1

You're probably coming up on the distinction between bandwidth and latency.

Latency is the time taken for a single round-trip, whereas bandwidth is the amount of data that can flow through over a given time period (eg 1 second).

If you're running 200 queries (from client-side code, not from a stored proc), then no matter how much data goes in each query, you will get 200 round-trips

Normal latency for the other end of the world is around half a second I believe - so for 200 entities retrieved separately, about 100 seconds.

Those numbers don't quite match yours, so there may be even higher latency (depends on all sorts of network factors). I would normally look for query/lookup overhead on the database server (assuming there is an indexing issue), but you've already mentioned that locally there is no significant overhead (presumably with the same data?).

Answer 2

You're noticing network latency .

There needs to be at least one round-trip with the server each time you make a query.

If the server is "far", ie 500ms ping, that means at minimum one second of latency per query. This is uncompressible - even if the query returns no rows, that 1s hit will happen.

The bandwidth of the network is an unrelated characteristic. If your bandwidth is high, you wont notice a big difference between transferring a large dataset and a small one. But both will still suffer the latency hit in exactly the same way.

I found this article has interesting (if dated) information: It's the latency, Stupid .