本地HTTPS代理可能吗？

Question

TL;DR

I want to set up a local HTTPS proxy that can ( LOCALLY ) modify the content of HTML pages on my machine. Is this possible?

Motivation

I have used an HTTP Proxy called GlimmerBlocker for years. It started in 2008 as a proxy-based approach to blocking ads (as opposed to browser extensions or other OS X-specific hacks like InputManagers). But besides blocking ads, it also allows the user to inject their own CSS or JavaScript into the page. Development has seriously slowed, but it remains incredibly useful.

The only problem is that it doesn't do HTTPS ( from its FAQ ):

Ads on https pages are not blocked

When Safari fetches an https page using a proxy, it doesn't really use the http protocol, but makes a tunneled tcp connection so Safari receives the encrypted bytes. The advantage is that any intermediate proxies can't modify or read the contents of the page, nor the URL. The disadvantage is, that GlimmerBlocker can't modify the content. Even if GlimmerBlocker tried to work as a middleman and decoded/encoded the content, it would have no means of telling Safari to trust it, nor to tell Safari if the websites certificate is valid, so Safari would think you have visited a dubious website.

Fortunately, most ad-providers are not going to switch to https as serving pages using https are much slower and would have a huge processing overhead on the ad-providers servers.

Back in 2008, maybe that last part was true…but not any more.

To be clear, I think the increasing use of SSL is a good thing. I just want to get back the control I had over the content after it arrives on my end.

---

Points of Confusion

While searching for a solution, I've become confused by some apparently contradictory points.

(Also, although I'm quite experienced with the languages of web pages, I've always had a difficult time grokking networks and protocols. On that note, sorry if I'm missing something that is way obvious!)

• I found this StackOverflow question asking whether HTTPS proxies were possible. The best answer says that “TLS/SSL (The S in HTTPS) guarantees that there are no eavesdroppers between you and the server you are contacting, ie no proxies.” (The same answer then described a hack to pull it off, but I don't understand the instructions. It was very theoretical, anyway.)

• In OS X under Network Preferences ▶︎ Advanced… ▶︎ Proxies , there is clearly a setting for an HTTPS proxy. This seems to contradict the previous statement that TLS/SSL's guarantee against eavesdropping implies the impossibility of proxies.

Other things of note

• I can't remember where, but I read that it is possible to set up an HTTPS proxy, but that it makes HTTPS pointless (by breaking the secure communication in the process). I don't want this! Encryption is good. I don't want to filter anyone else's traffic; I just want something to customize the content after I've already received it.

• GlimmerBlocker has a nice GUI interface, but I'm fine with non-GUI solutions, too. I may have a poor understanding of networking and protocols, but I'm perfectly comfortable on the command line, tweaking settings in text editors, and so on.

---

Is what I'm asking possible? Or is my question a case of “either you get security, or you can break it with hacks and get to customize your content—but not both”?

Answer 1

The common idea of a HTTP proxy is a server which accepts a CONNECT request which includes the target hostname and port and then just builds a tunnel to the target server. All the https is done inside the tunnel, so there is no way for the proxy to modify it (end-to-end security from browser to web server).

To modify the data you need to have a proxy which plays man-in-the-middle. In this case you have a https connection between the proxy and the web server and another https connection between the browser and the proxy. Between proxy and web server the original server certificate is used, while between browser and proxy a newly created certificate is used, which is signed by a CA specific to the proxy. Of course this CA must be imported as trusted into he browser, otherwise it would complain all the time about possible attacks.

Of course - all the verification of the original server certificate has to be done in the proxy now, and not all solutions do this the correct way. See also http://www.secureworks.com/cyber-threat-intelligence/threats/transitive-trust/

There are several proxy solution which might do this SSL interception, like squid, mitmproxy (python) or App::HTTP_Proxy_IMP (perl). The last two are specifically designed to let you modify the content with your own code, so these might be good places to start.