How do I read the header without reading the rest of the HTTP resource?

Question

I was sure that when I do like this:

http = Net::HTTP.start uri.host, uri.port
request = Net::HTTP::Get.new uri
response = http.request request

I get some sort of established connection to the remote HTTP resource to be able to tell, for example, its Content-Type, while to really load the whole resource I then call the response.body .

But either I was always wrong or it is something with the server I access right now, the http.request loads the whole remote file that is unacceptable for me:

[Net::HTTP debug] opening connection to v.redd.it:80...  
[Net::HTTP debug] opened  
[Net::HTTP debug] <- "GET /6otzwem1c7721/DASH_9_6_M?source=fallback HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nUser-Agent: Ruby\r\nHost: v.redd.it\r\nConnection: close\r\n\r\n"  
[Net::HTTP debug] -> "HTTP/1.1 200 OK\r\n"  
[Net::HTTP debug] -> "Last-Modified: Sat, 29 Dec 2018 11:25:57 GMT\r\n"  
[Net::HTTP debug] -> "ETag: \"662291aec20b252aaebcf54c3b1827af-42\"\r\n"  
[Net::HTTP debug] -> "Content-Type: video/mp4\r\n"  
[Net::HTTP debug] -> "Cache-Control: public, max-age=604800, s-maxage=86400, must-revalidate\r\n"  
[Net::HTTP debug] -> "Accept-Ranges: bytes\r\n"  
[Net::HTTP debug] -> "Content-Length: 218756120\r\n"  
[Net::HTTP debug] -> "Accept-Ranges: bytes\r\n"  
[Net::HTTP debug] -> "Date: Sun, 30 Dec 2018 13:44:21 GMT\r\n"  
[Net::HTTP debug] -> "Via: 1.1 varnish\r\n"  
[Net::HTTP debug] -> "Connection: close\r\n"  
[Net::HTTP debug] -> "X-Served-By: cache-fra19120-FRA\r\n"  
[Net::HTTP debug] -> "X-Cache: HIT\r\n"  
[Net::HTTP debug] -> "X-Cache-Hits: 0\r\n"  
[Net::HTTP debug] -> "X-Timer: S1546177461.284280,VS0,VE0\r\n"  
[Net::HTTP debug] -> "Server: snooserv\r\n"  
[Net::HTTP debug] -> "Vary: Origin\r\n"  
[Net::HTTP debug] -> "\r\n"  
[Net::HTTP debug] reading 218756120 bytes... 

I went inside with byebug until found where it happens:

[159, 168] in /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/net/http/response.rb
   159:   def reading_body(sock, reqmethodallowbody)  #:nodoc: internal use only
   160:     @socket = sock
   161:     @body_exist = reqmethodallowbody && self.class.body_permitted?
   162:     begin
   163:       yield
=> 164:       self.body   # ensure to read body
   165:     ensure
   166:       @socket = nil
   167:     end
   168:   end
(byebug) where
--> #0  Net::HTTPResponse.reading_body(sock#Net::BufferedIO, reqmethodallowbody#TrueClass) at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/net/http/response.rb:164
    #1  Net::HTTP.transport_request(req#Net::HTTP::Get) at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/net/http.rb:1445
    #2  Net::HTTP.request(req#Net::HTTP::Get, body#NilClass, &block#NilClass) at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/net/http.rb:1407
    #3  block in Net::HTTP.block in request(req#Net::HTTP::Get, body#NilClass, &block#NilClass) at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/net/http.rb:1400
    #4  Net::HTTP.start at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/net/http.rb:853
    #5  Net::HTTP.request(req#Net::HTTP::Get, body#NilClass, &block#NilClass) at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/net/http.rb:1398

Is it server's fault? Ruby fault? Or should I use some another method if I want to get the header data without loading the whole resource?

PS: I do not need third-party fancy dependency, I need to use only the Net::HTTP .

Answer 1

Depending on what you actually want to accomplish:

Don't care about the body at all:

Use HEAD instead of GET :

uri = URI('http://example.com')
http = Net::HTTP.start uri.host, uri.port 
request = Net::HTTP::Head.new uri
response = http.request request
response.body
# => nil

Load the body conditionally

Using blocks with net/http will let you hook before the body is actually loaded:

uri = URI('http://example.com')
res = nil

Net::HTTP.start(uri.host, uri.port) do |http|
  request = Net::HTTP::Get.new uri

  http.request request do |response|
    res = response
    break
  end
end

res
# => #<Net::HTTPOK 200 OK readbody=false> 
res['Content-Type']
# => "text/html; charset=UTF-8"