Right now the CPython core development team are having an extremely lengthy conversation about whether to enable certificate verification for HTTPS clients. When it comes to making this the new default behaviour in Python 3.5 the response is an enthusiastic “Yes!” (just as it should be). However, when it comes to backporting that behaviour to a patch release of Python 2.7 the response from the core committers is distinctly more lukewarm.
Stating this up front, I’m strongly in favour of backporting the fix to Python 2.7. Yes, it will break people’s code, but guess what: that code was already broken! The user’s just didn’t know it yet.
Here’s the thing: HTTPS has semantic meaning. It’s not just “HTTP with TLS”. The desire to do that is being addressed by the HTTPBis Working Group in the Opportunistic Security Internet Draft. HTTPS carries significantly more baggage than just purely encryption.
The covenant of HTTPS is this: when you ask for a website with a URL that has
the scheme HTTPS you are asking to contact someone who is authoritative for
that resource. You are essentially cutting all intermediaries out of your
connection in an attempt to get exactly the resource you asked for.
This means that validating the certificate isn’t incidental to HTTPS, it is vital to HTTPS. If you present a certificate that is not valid for the domain that I’m trying to contact, then I should absolutely stop contacting you. I should fail hard. I should throw my toys out of the pram and totally refuse to collect them again.
What I should not do is shrug my shoulders and go ahead anyway. Anyone
relying on the fact that httplib or http.client or urllib2 all do exactly
that is either being lied to by their client or, just as bad, they are
mis-using HTTPS.
Yes, there are organisations in the world that provide resources via HTTP over TLS on port 443 without valid certificates. Those organisations are wasting their CPU cycles on encrypting data that is clearly of no importance to them. Worse, they are implying a level of privileged communication to both ends of the connection that neither end has. Not verifying the certificate doesn’t just put the client at risk: it puts the server at risk as well. The server can’t tell you didn’t verify the certificate, and so it can’t tell whether you got MITM’d or not.
Yes, there are problems with certificates and certificate authorities that make a lot of people unhappy with HTTPS. Tough. Want to use self-signed certs? Fine, but either add the cert to your trust store or don’t call what you’re doing HTTPS: it isn’t.