HttpsProxy

Musings on writing a proxy for Secure HyperText Transport Protocol (HTTPS).

Overview

HTTPS is supposed to be "secure", meaning that there is a secure connection between the HTTPS client (typically a web-browser) and the HTTPS server.

Because this connection is designed and intended to be secure, HTTPS is difficult to "proxy", or to place a so-called "man-in-the-middle". However, the protocol only requires the client to authenticate (prove the identity of) the server - the server is not required to, nor can it, prove the identity of the client in order to correctly implement HTTPS.

The security relies on Secure Socket Layer (SSL), or Transport Layer Security (TLS), which is established at connection set-up time. The HTTPS client requests an SSL certificate from the server which has been signed by a certificate signing authority. If the server replies with a certificate containing the correct credentials and signed by a trusted authority, then the client can proceed to establish an encrypted session to the server.

In order to proxy HTTPS, the trick is to place a locally-generated root certificate in the client that matches a signing certificate in the proxy, then getting the proxy to sign all certificates it passes back to the HTTPS client in such a way that the client can authenticate the proxy.

Background

HTTPS requests can be directed to a proxy, if the browser has been set up with an SSL proxy entry. In this case, the browser will start of will a CONNECT request to the proxy, before negotiating an SSL connection directly with the target secure server. Otherwise, the HTTPS requests will need to be Transparently proxied.

A HTTPS client (eg. a web browser) will start off loading a web page over HTTPS via proxy by:

1. opening a TCP connection to the SSL port of the proxy
2. client sends an HTTPS CONNECT request, specifying the DNS name of the target secure server
3. proxy determines target servers IP address and opens a connection to it
4. proxy sends back a "Connection Established" message to client
5. proxy now just pushes packets back and forth between client and server
6. client requests servers SSL certificate (via proxy)
7. server provides certificate (via proxy)
8. client checks that certificate is signed by a recognised (to the client) certificate signing authority
9. if OK, client and server exchange session keys (via proxy)
10. client sends SSL-encrypted HTTP request to server (via proxy)
11. server responds with SSL-encrypted HTTP response (via proxy)

Note that HTTP v1.1 allows multiple requests from the client down the same connection to the same server, in this case, via the proxy.