Stream large http response to disk in Erlang with httpc lib

Question

httpc:request(get, {URL, []}, [], [{stream, FilePath}]) returns {ok, saved_to_file} even when the remote web-service dies midway while streaming the response. I checked the content on the disk was missing a whole bunch of data. Is this a known quirk and expected?

Answer 1

I filed a bug-report and the answer from OTP team indicated it was a bug (atleast for the current HTTP spec):

I think this is due to that the client has backwards compatibility with servers that would close the connection to indicate that all data has been sent. Probably this should be made conditional for only old versions of the HTTP protocol or maybe it could be dropped... - Ingela Anderton Andin