The Grumpy Troll

Ramblings of a grumpy troll.

TLS, CRIME, BEAST and you the programmer

Before continuing: I am not a cryptographer; merely someone who, on a good day, can use cryptographic libraries with what might pass for intelligence. I write this post with 20/20 hindsight.

One of the security maxims I hold dear is to not freely mix data from different security contexts. Programmers often see this with “don't mix code and data”, and guidance to prepare parameterised SQL queries and then execute them with untrusted data in parameters, rather than try to sanitise the data into an SQL query. Many website security flaws arise from a failure to follow this maxim.

Both the BEAST attack, and this new CRIME (Compression Ratio Info-leak Made Easy) attack, impressively manage to attack how common use of HTTP within TLS abuses this maxim.

The core problem is that folks combine highly sensitive data, such as authentication cookies or Authorization: HTTP headers with OAuth2 bearer tokens, with a POST payload (or anything with a client payload, PUT, PATCH, etc; [EDIT: actually, this time GET works, the attack can be in the path]) that might contain untrusted data. TLS protects one stream of data assumed to be of a homogenous security level, and is not designed to protect the "cleartext" against itself. So in practice, with browsers with retained authorisation information and web-sites protecting against cross-domain attacks, HTTPS is using HTTP over TLS in a way which TLS was not designed to protect. Nonetheless, TLS does a fair job of it: so much so, that folks are lulled into believing they're safe.

So both BEAST and now CRIME work by seeing how varying a later part of one stream can reveal information about an earlier part of the stream. [EDIT: actually, CRIME can reveal later parts of a stream too, it is bi-directional]

In BEAST's case, the unfortunate way that TLS historically specified choosing the initialisation vector for each block in the CBC mode of using a block cipher to protect a stream led to disclosure. Using a different cipher mode, or using a modern version of TLS, helps with that. Fortunately, the dominant SSL framework (OpenSSL) now has a stable release out which supports said modern versions of TLS, so folks have been busily upgrading to protect themselves and their users. See Wikipedia's “Block cipher modes of operation” for an introduction to modes, CBC, and my favourite demonstration image of why naive block cipher usage (“ECB mode”, aka no mode) is a really bad idea.

In CRIME's case, compression is used to attack TLS. Normally, folks have been advised to use compression to increase the per-octet entropy of the cleartext, which may help make cryptographic attacks harder. This still holds true, if the cleartext is all of one security context. If instead the payload can be modified by an attacker, across repeated requests, then when a pattern in the payload matches a pattern in the sensitive authorisation data, earlier in the stream, then compression will do what it's designed to do, and produce a smaller stream. That change in size is visible after encryption, so someone who can observe the TLS session can adjust their payload and gradually discern more of what the authorisation data must be, without directly decrypting it.

As long as sites use HTTPS with authorisation data inside the cleartext, problems like this will recur. They'll be band-aided, often by reducing the functionality of TLS, but they'll occur. The authorisation data needs to move out of the cleartext and instead be expressed in some other way. Perhaps issuing a short-lived client TLS certificate which will be accepted by the site. This affects cookies, OAuth2 with bearer tokens, and more.

This does not affect stand-alone client applications which do not let their credentials be used for all remote access, including from untrusted sources; it does affect them if they register a URL handler in a client's system and they suddenly start taking untrusted requests.

This does not affect protocols which do not mix authentication or authorisation data with untrusted data, to result in data flows which will not be visible to the user of the application.

CRIME does not affect TLS sessions in which compression is not used. Compression is part of the core protocol, but is optional. Programs written in the Go programming language, for instance, are likely not vulnerable simply because the only compression mechanism supported by crypto/tls is (currently) None. If, though, they use a Go wrapper for an external security library, then this is not relevant and they may still be vulnerable.

Browsers have been disabling compression, which prevents this attack but across many types of network path will result in slower https access.

I suspect that the longer-term work-around for CRIME will be to require the sender of mixed-security-level content to coerce a compression block end after the HTTP headers and before the untrusted payload. If the DOM API permits requests to be sent with arbitrary headers under the attacker's control, then move those headers to the end of the header and insert an extra block switch there. For DEFLATE compression (RFC 1951), the coding tree for each block is independent; if a compression scheme maintains a coding tree across blocks, then they'll also need a reset of the coding tree, since that's what really matters. [EDIT: or rather, move those untrusted headers to the start of the headers, because the attack can be in a GET line, not just in a payload]

Before now, I've not looked too closely at how compression and TLS libraries interact, so this paragraph is based on some quick API and code analysis. For ZLib itself, it appears that Z_FULL_FLUSH will be needed for calls to deflate(). Both OpenSSL and GnuTLS normally call deflate(..., Z_SYNC_FLUSH) to let a compression block span multiple network-layer sends.

For OpenSSL, this full flushing requirement boils down to a call to BIO_flush() between security contexts. Note that BIO_flush() is not guaranteed to end a compression block, but in practice does. Ideally, there would be a way to cork the actual network-layer send across the two or three upper-level flushes, before uncorking to send.

I do not see a way to force a block end in GnuTLS (checking current git master branch).

An earlier analysis of mine into how BEAST affects SMTP and Exim in particular is available in the exim-announce archives. The same principles apply to CRIME.

See also: Thomas Mornin on the Security StackExchange

Regards,
-The Grumpy Troll
Categories: TLS GnuTLS security OpenSSL BEAST zlib OAuth2 deflate HTTPS authentication authorisation golang CRIME compression