How to change URL timeout settings on linux webserver

Question

I have some cron job set up on Linux through wget , those jobs run once every 24 hours. All the jobs are basically calling the APIs, pulling the data and I am strong it on the database. Now the issue is some API calls are very very slow and takes a lot of time to get a response that eventually ends up getting the below error.

--2017-07-24 06:00:02-- http://wwwin-cam-stage.cisco.com/cron/mh.php Resolving wwwin-cam-stage.cisco.com (wwwin-cam-stage.cisco.com)... 171.70.100.25 Connecting to wwwin-cam-stage.cisco.com (wwwin-cam-stage.cisco.com)|171.70.100.25|:80... connected. HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers. Retrying.

--2017-07-24 06:05:03-- (try: 2) http://wwwin-cam-stage.cisco.com/cron/mh.php Connecting to wwwin-cam-stage.cisco.com (wwwin-cam-stage.cisco.com)|171.70.100.25|:80... connected. HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers. Retrying.

--2017-07-24 06:10:05-- (try: 3) http://wwwin-cam-stage.cisco.com/cron/mh.php Connecting to wwwin-cam-stage.cisco.com (wwwin-cam-stage.cisco.com)|171.70.100.25|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 0 [text/html] Saving to: 'mh.php.6'

  0K 0.00 =0s 

2017-07-24 06:14:58 (0.00 B/s) - 'mh.php.6' saved [0/0]

Though at third try it gave response 200 OK, it messes up the actual data as it timed out in a first and second try.

How can I change the URL timeout settings to unlimited time or highest limit in order to complete job in one try and without getting error like

(Connection reset by peer)....

Answer 1

wget --timeout 10 http://url 

This can be used in case of wget .

EDIT

Or

If you are asking about the Keep-Alive of linux machine this might help.

On RedHat Linux modify the following kernel parameter by editing the /etc/sysctl.conf file, and restart the network daemon ( /etc/rc.d/init.d/network restart ).

"Connection reset by peer" is the TCP/IP equivalent of slamming the phone back on the hook. It's more polite than merely not replying, leaving one hanging. But it's not the FIN-ACK expected of the truly polite TCP/IP converseur.

Code:

# Decrease the time default value for tcp_keepalive_time
tcp_keepalive_time = 1800 

EDIT

-T seconds

‘--timeout=seconds’

Set the network timeout to seconds seconds. This is equivalent to specifying '--dns-timeout' , '--connect-timeout' , and '--read-timeout' , all at the same time.

When interacting with the network, Wget can check for timeout and abort the operation if it takes too long. This prevents anomalies like hanging reads and infinite connects. The only timeout enabled by default is a 900-second read timeout. Setting a timeout to 0 disables it altogether. Unless you know what you are doing, it is best not to change the default timeout settings.

All timeout-related options accept decimal values, as well as subsecond values. For example, '0.1' seconds is a legal (though unwise) choice of timeout. Subsecond timeouts are useful for checking server response times or for testing network latency.

‘--dns-timeout=seconds’

Set the DNS lookup timeout to seconds seconds. DNS lookups that don't complete within the specified time will fail. By default, there is no timeout on DNS lookups, other than that implemented by system libraries.

‘--connect-timeout=seconds’

Set the connect timeout to seconds seconds. TCP connections that take longer to establish will be aborted. By default, there is no connect timeout, other than that implemented by system libraries.

‘--read-timeout=seconds’

Set the read (and write) timeout to seconds seconds. The “time” of this timeout refers to idle time: if, at any point in the download, no data is received for more than the specified number of seconds, reading fails and the download is restarted. This option does not directly affect the duration of the entire download.

Of course, the remote server may choose to terminate the connection sooner than this option requires. The default read timeout is 900 seconds.

Source Wget Manual.

See this wget Manual page for more information.