During the interview process for my current role of Developer Advocate for API security I was asked to do an informal presentation about HTTPS, and to be sure I would get the role I went deep the rabbit hole and took as many notes as I could in this repo, that I am now making available as a blog post.
This is not an exhaustive list of everything I know about HTTPS, but its my intention to keep updating it with more information as my free time allows. I hope this can become a good starting point and/or reference for developers of any level of experience.
TOC - Table of Contents
- HTTPS Benefits
- HTTPS vs HTTP Speed
- TLS Handshake
- HTTPS Prevents Man in the Middle Attacks
- HTTPS and Headers
- Content Links
- Brotli Compression
- HTTPS Acronyms
- Confidentiality - it keeps our info protected, once is always encrypted during all the communication process.
- Integrity - we know the content that is being loaded is the original one, by other words has not been modified.
- Authenticity - SSL certificates guarantees that we are communicating with the original server, not a malicious one.
- SEO - Better SEO ranking on Google for sites using HTTPS.
- Brotli Compression - less size in content leads to speed improvements.
- HTTP 2.0 protocol - allows parallel downloads instead of cascaded ones that translates to great speed improvements and this protocol is only available over HTTPS.
HTTPS vs HTTP Speed
httpvshttps.com website we can see that the HTTP version takes around 12 seconds to load all images and the HTTPS version only takes around 1 second. The results will vary with your internet speed, but the magnitude of the difference will be noticeable.
Wow, this is amazing, but how can that be possible?
If we open the developer tools in the browser and enable the protocol column on the network tab then we will see that on the HTTP version of the website the images are loaded hover HTTP 1.1 protocol while in the HTTPS version they are loaded hover HTTP 2.0 protocol.
On HTTP 1.1 the browser is loading images in cascade, once the number of requests per connection to the server are limited, but in HTTP 2.0 protocol this limitation is not present, therefore all images can be requested over the same connection, by other words they are downloaded in parallel instead of in cascade.
But to get this boost in speed the server must support HTTP 2.0 protocol and this protocol is only available in some more recent browsers.
Browser support for HTTP 2.0 can be checked on the caniuse.com website.
If we keep our website in HTTP we will not be able to take advantage of HTTP 2.0 protocol and its incredible boost in speed.
So, HTTPS not only gives us security but also gives us speed :).
During the TLS handshake:
- Client communicates the level of TLS it supports.
- Client communicates the Cipher Suites it supports in the order it prefers to use.
- Server agrees with the client in what level of TLS and Cipher Suites to be used.
- Server provides is public key to the client.
- Client verifies the server public key against is local list of Certificate Authorities(CAs).
- Once verified the public key Client will be confident that is talking with the correct server.
- All previous communication is called the negotiation phase and is not encrypted, but no content is exchanged. This phase is visible in a MitM(Man in the Middle) attack.
- Client exchanges is key with the server, but sends it encrypted with the previous received server public key.
- Server finishes the handshake and the secure communication to exchange content can start.
HTTPS Prevents Man in the Middle Attacks
Man in the middle (MitM) attack is possible when the communication between 2 endpoints can be intercepted and modified, due to the lack of using the HTTPS secure protocol.
Using HTTPS will prevent this attacks to happen, once the attacker can’t see the content being sent over the wire in plain text, like he can with HTTP.
Bear in mind that a MitM attack is still possible to perform when HTTPS is used. For example, when the attacker controls the device being used or when is able to trick users to install the self-signed certificates from the MitM tool being used.
Professionally I have wrote several blogs on the subject that I invite you to read:
Examples of when it can happen over HTTP
- when a device connects to a wifi router.
- compromised wifi router.
- when the wifi router connects to the ISP using HTTP.
- ISP sniffing due to law enforcement.
- when the ISP connects to the destination several exchange points will be reached and state agents can be sniffing the traffic, like governments.
Examples on how they can happen:
- session hijacking due to the use of unprotected cookies, like in Facebook and Twitter cases, by using a tool like Firesheep.
- loading web forms hover HTTP, even when they post to HTTPS, because they may have been modified by a MitM attack. For example, they may have the post URL modified to target a malicious web server, instead of the one serving the website.
- malicious/unauthorized tracking of our activity, by injecting things into the content we send/receive. Some ISPs do this.
- injected malware in a compromised router/isp or any other exchange point. Like the Tunisia Keystroke logger inject on Facebook, Gmail and Yahoo login pages loaded hover HTTP.
- DNS hijacking by compromising our DNS resolvers to serve the site from a server they control, making this way possible for them to change/replace the original content when the site is served hover HTTP. If the site is being served hover HTTPS, than to serve their own content they also need the valid TLS certificate for the site.
- routers compromised - like here, here and here.
HTTPS Helps to mitigate:
- phishing attacks - because when we click in a fake link to a website served hover HTTPS, they can’t spoof the TLS certificate for the original website.
- malicious host files - if the host file in our device is compromised and points a HTTPS website to a different IP address, the TLS certificate will not match, despite the URL in the browser being the correct one.
- Cross Site Request Forgery or CSRF - like on the routers compromised example given above.
HTTPS and Headers
HTTPS Strict Transport Security (HSTS)
This a header field we can set to instruct the browser to always make the initial and all subsequent requests to our website hover HTTPS.
# The max age value is in seconds. Strict-Transport-Security: max-age=63072000; includeSubdomains; preload
includeSubdomains policy means that all sub domains of the website will benefit of the same security policy.
preload policy means that the browser should include it on the local preload list for the device, but not into the HSTS Preload list shipped with the browser on installation, and that is then kept up to date each time the browsers receives an update.
When the website is not in the HSTS Preload list but uses the
preload policy and the user types
example.com instead of
https://example.com then the HSTS header to enforce the browser to always use HTTPS is fetched over an insecure HTTP connection, therefore subject to be manipulated in a MitM attack.
Having the backend server doing a redirect from HTTP to HTTPS is not enough to protect a user from having the request for
example.com manipulated or redirected in a MitM attack.
So, to protect the website from being vulnerable to this scenario is recommended to add it into the hstspreload.org list, and all the browsers with the updated list will always make a request to
https://example.com when the user just types
Browser support for the HSTS header can be checked at caniuse.com.
Content Security Policy (CSP)
Content-Security-Policy is the name of an HTTP header used by modern browsers to improve security when loading a web page. It can also be used as a meta-tag in the web page HTML.
To test the CSP header on your website you can use the securityheaders.com online tool, but bear in mind that this tool doesn’t read HTML meta tags.
Let’s see some brief examples of using a CSP policy.
Upgrade insecure requests
This CSP policy will instruct the browser to upgrade all request from HTTP to HTTPS in a web page.
When using the CSP header, the response to the request to fetch the web page must contain an header like:
When using the CSP meta tag, the web page HTML must contain something like this:
<html> <head> <meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests"> </head> </html>
This is useful to:
- ensure third party content embedded in our page is not loaded over HTTP, like Facebook, Twitter or Disqus comments.
- ensure that in case we forgot in our own code to load something over HTTPS it will be forced to use it.
Browser support can be checked on the caniuse.com website.
Block all mixed content
HTML meta tag example:
<html> <head> <meta HTTP-equiv="Content-Security-Policy" content="block-all-mixed-content"> </head> </html>
Browser support can be checked at caniuse.com.
When a user clicks in a HTTP link on a HTTPS page the referrer header will not be set in order to protect from potentially sending sensitive information in the HTTPS referrer link to the HTTP page, because if it was set then a MitM attack could easily obtain that sensitive information.
However the referrer header from an HTTPS page to another HTTPS page is always sent, meaning that sensitive information on it will be sent to the destination, therefore exposed/leaked to them.
So if the HTTPS destination page is not in our control we may not want the referrer header to be set and to control this we have some options:
- Content Security Policy Referrer Header
- The Referrer Policy Header
- URL Mask Service - This will hide the referrer.
- Shortening URL services should also work to hide the referrer, need to test it.
Support for the referrer policy header can be checked at caniuse.com.
Cookies must always be set with the
Secure flag enabled to prevent them to be sent over HTTP, preventing them from being easily accessed by unintended parties, like in a MitM attack.
httpOnly flag in the cookie.
We can read more about restricting access to cookies in this page from Mozilla developers.
As rule of thumb Relative URLs should be only used for external links and for internal links we should always use Relative Paths.
Internal link is any link referencing anything under the same domain we served the content.
External link is any link referencing any link not belonging to the current domain, even a sub domain of the domain serving the content.
example.com as the one used to serve the content, than
api.example.com is considered a external link, but
example.com/api is an internal link.
When we use external links in our content we normally use the absolute URL like
http://external-link.com, but this is coupling us to a specific URL scheme, the HTTP one an when we implement HTTPS we will need to go back to our code and find all this URLs to fix them.
A best approach is to use a Relative URL where the URL scheme is omitted, therefore the browser prepends the URL scheme used to serve the content any time it needs to use this link.
So instead of using:
http://external-link.comwe should use
//external-link.com will be interpreted by the browser as:
http://external-link.comwhen content is served over HTTP.
https://external-link.comwhen content is served over HTTPS.
When serving the content with local assets or linking to internal resources is preferable to use the relative path instead of the relative URL.
Every time the browser finds a relative path it prepends the URL used to serve the content before trying to use it.
So instead of using:
//example.com/css/global.csswe should use
//example.com/users/1we should use
http://example.com/css/global.csswe should use
http://example.com/users/1we should use
css/global.css will be interpreted by the browser as:
http://example.com/css/global.csswhen content is served over HTTP.
https://example.com/css/global.csswhen content is served over HTTPS.
This is a new compression algorithm only available over HTTPS, that was introduced by Google in 2015.
This algorithm improves compression by around 25%.
A more in depth article by Scott Helme can be found here.
Explaining the meaning of some acronyms used when talking about HTTPS implementations.
CAA - Certificate Authority Authorization
At the DNS level we can specify what Certificate Authorities we allow to issue certificates for the domain.
This also protects us from Certification Authorities that have been compromised, like the already mentioned DigiNotar Iran case.
CRL - Certificate Revocation List
This is a list that each Certification Authority needs to maintain to list all is revoked certificates, so that all clients using it can know what certificates they can’t trust any more.
So, this is a list that will be shipped with the browser or with the Operating System, therefore making the revocation process slow and dependent on our system updates.
DANE - DNS Based Authentication of Named Entities
Used to specify at DNS level the certificate keys being used. DNSSEC must be enabled to use DANE.
This helps to protect the compromise of a Certificate Authority, like the famous DigiNotar Iran case.
A brief explanation how they Certificate Authorities can be attacked can be found here.
DNSSEC - Domain Name System Security Extensions
Prevents the forgery of DNS records for the domain, guaranteeing this way the integrity of the Domain.
The forgery of DNS records allows an attacker to route the traffic to their own servers to what so ever they want with it.
HPKP - HTTP Public Key Pinning
Defines the certificate public keys that are admissible by the client for each domain.
In is attributes we can:
- list all the public keys for allowed the certificate.
- specify the max age.
- tell if it applies to sub domains.
- provide the URL to report violations for attempts on use of invalid public keys.
This is implemented in the response with the
A draw back of this is that we need to have a first trusted request, so that subsequent calls to the domain can be protected from man in the middle attacks.
OCSP - Online Certificate Status Protocol
An alternative to CRL where the Certificate Authorities implement the OCSP protocol to handle almost in real time the certificates revocation.
OCSP can cause some privacy issues, therefore OCSP Stapling is preferred.
In a OCSP implementation the website is responsible to validate the certificate by calling the OCSP server of the Certification Authority and return the result in the TLS hand shake between server and client.
PFS - Perfect Forward Secrecy
In case a private key for the certificate is compromise it protects any past sessions encrypted with that private key to become also compromised.
PKP - Public Key Pinning
The client will only accept certificates from a predefined list with the public keys of the certificates it can trust on.
This by itself does not guarantee to the client that a certificate in the predefined list is the correct one for the domain… Remember the DigiNotar Iran case.
SAN - Subject Alternative Name
Makes possible to use multiple domain names in the same certificate.
SNI - Server Name Indication
Enables multiple certificates in the same IP address.
SSL - Secure Sockets Layer
SSL was commonly used until 2014 Poodle attack to implement HTTPS but is now suppressed by TLS.
TLS - Transport Layer Security
Nowadays TLS is the standard to implement HTTPS, once is more secure and fast and with it the Poodle attack would havenot succeeded.
- Automated Installation of Certificates - Automates the process of acquire/renew/install Let’s Encrypt certificates.
- Mozilla Observatory - The most complete set of tools to test the website security.
- SSL Server Test - The famous Qualys SSL Labs tester.
- Security Headers Checker
- Real Time Security Reporting
- Fiddler - Proxy to debug HTTP and HTTPS.
- BadSSL - site to show all that can go wrong with an HTTPS implementation.
- HSTS Preload - verifies if a site is in the browsers HTTPS preload list, if they are eligible to be there or to submit a site to this preload list.
- Chromium Preload List - List of all preloaded sites.
- Browser Compatibility Checker for HSTS.
- HTTP Vs HTTPS Speed - Compare speed of loading this website hover HTTP vs HTTPS connection.
- Lets’s Encrypt - Let’s Encrypt gives us certificates for free with a 3 months expiring period.
- Troy Hunt - What Every Developer Should Know About HTTPS
- SSL Labs Wiki
- Open SSL CookBook
- Bullet Proof SSL and TLS
- Is TLS Fast Yet - website about speed impact when using secure communications.
- DigiNotar CA Compromised - The history of how Iran compromised this CA to spy on is citizens email accounts.
- Certificate Authorities Attacks - A brief explanation how they can get compromised.
Disclaimer: What I expressed here is only in my behalf and doesn't represent the company I work for, or any previous one, neither my family, friends, colleagues or anyone else unless I explicitly say so.