What happens when the python thread is in network call(HTTPS) and the context switch happens?

Question

I am having a problem where a multithreaded download application that we wrote is sometimes getting 408 error. We think it might be because the user has increased the number of threads,and this has caused the error. We think it may be possible that during the network call, context switch happens and that results in thread not sending all the packets required for the call to succeed and sever gives a 408 timeout error. Is that possible or is it that network calls are not dependent with context switches.

We are using python threads and pycurl module to download the data using 120 threads.

Answer 1

What context switch are you referring to?

Pycurl is a wrapper around libcurl which is a C library. When any of libcurl's methods are called, Python's global interpreter lock is released. Python runtime may run its own threads while libcurl is performing client-side processing, or during actual network I/O, but "context switches" aren't attached to network I/O in any way.

This shouldn't affect your troubleshooting however because you should be using a packet capture tool like tcpdump to record your traffic and identify which side is sending what when you are getting the errors so that you can make some reasoned theories as to what might be going on instead of guessing.