HTTPS through HTTP proxy

Question

I have service which works like a proxy, you can get web pages through it. For example via telnet

GET http://example.com HTTP/1.1
Host: example.com

But if I want download https page I should do the following

GET https://example.com HTTP/1.1
Host: example.com
Https-Header: true

And I want to write scala client for this service using apache http client, using service like a proxy host.

private val DefaultProxy = new HttpHost("service host", port)
private val DefaultClient =
HttpClientBuilder.create().
  setProxy(DefaultProxy).
  build()

I can successfully download http pages, but when I try to download https pages, apache client makes CONNECT request to the proxy, and it response with error, cause service can operate only with GET requests. How can I make apache client work with https pages like with http, that's mean send GET request to proxy, not CONNECT?

Answer 1

To download an https webpage in the same way than an http one with telnet you need to establish the ssl/tls connection first:

openssl s_client -connect www.somesite:443

[watch the ssl certificate details scroll by]

GET /index.html HTTP/1.1

Host: www.somesite

Example from https://www.bearfruit.org/2008/04/17/telnet-for-testing-ssl-https-websites/

For scala maybe that can help you : https://github.com/scalaj/scalaj-http

HTTPS is HTTP over SSL/TLS so you need something to establish the SSL/TLS secure tunnel to the website, then you can send your HTTP request.

Answer 2

I find out a solution.

I write custom HttpRoutePlanner which always provide not secure route, and then Apache client work with https link like with http link, there is a HttpRoutePlanner code

private def routePlanner(proxy: HttpHost) = new HttpRoutePlanner() {

  def determineRoute(target: HttpHost ,
                     request: HttpRequest,
                     context: HttpContext) = {
    new HttpRoute(target, null,  proxy, false)
  }
}