All I Know About HTTPS

Sun, 16 May 2021 HTTPS, security, TLS, SSL

During the interview process for my current role of Developer Advocate for API security I was asked to do an informal presentation about HTTPS, and to be sure I would get the role I went deep the rabbit hole and took as many notes as I could in this repo, that I am now making available as a blog post.

This is not an exhaustive list of everything I know about HTTPS, but its my intention to keep updating it with more information as my free time allows. I hope this can become a good starting point and/or reference for developers of any level of experience.

HTTPS Benefits

Confidentiality - it keeps our info protected, once is always encrypted during all the communication process.
Integrity - we know the content that is being loaded is the original one, by other words has not been modified.
Authenticity - SSL certificates guarantees that we are communicating with the original server, not a malicious one.
SEO - Better SEO ranking on Google for sites using HTTPS.
Brotli Compression - less size in content leads to speed improvements.
HTTP 2.0 protocol - allows parallel downloads instead of cascaded ones that translates to great speed improvements and this protocol is only available over HTTPS.

TOC

HTTPS vs HTTP Speed

Using the httpvshttps.com website we can see that the HTTP version takes around 12 seconds to load all images and the HTTPS version only takes around 1 second. The results will vary with your internet speed, but the magnitude of the difference will be noticeable.

Wow, this is amazing, but how can that be possible?

If we open the developer tools in the browser and enable the protocol column on the network tab then we will see that on the HTTP version of the website the images are loaded hover HTTP 1.1 protocol while in the HTTPS version they are loaded hover HTTP 2.0 protocol.

On HTTP 1.1 the browser is loading images in cascade, once the number of requests per connection to the server are limited, but in HTTP 2.0 protocol this limitation is not present, therefore all images can be requested over the same connection, by other words they are downloaded in parallel instead of in cascade.

But to get this boost in speed the server must support HTTP 2.0 protocol and this protocol is only available in some more recent browsers.

Browser support for HTTP 2.0 can be checked on the caniuse.com website.

If we keep our website in HTTP we will not be able to take advantage of HTTP 2.0 protocol and its incredible boost in speed.

So, HTTPS not only gives us security but also gives us speed :).

TOC

TLS Handshake

During the TLS handshake:

Client communicates the level of TLS it supports.
Client communicates the Cipher Suites it supports in the order it prefers to use.
Server agrees with the client in what level of TLS and Cipher Suites to be used.
Server provides is public key to the client.
Client verifies the server public key against is local list of Certificate Authorities(CAs).
Once verified the public key Client will be confident that is talking with the correct server.
All previous communication is called the negotiation phase and is not encrypted, but no content is exchanged. This phase is visible in a MitM(Man in the Middle) attack.
Client exchanges is key with the server, but sends it encrypted with the previous received server public key.
Server finishes the handshake and the secure communication to exchange content can start.

TOC

HTTPS Prevents Man in the Middle Attacks

Man in the middle (MitM) attack is possible when the communication between 2 endpoints can be intercepted and modified, due to the lack of using the HTTPS secure protocol.

Using HTTPS will prevent this attacks to happen, once the attacker can’t see the content being sent over the wire in plain text, like he can with HTTP.

Bear in mind that a MitM attack is still possible to perform when HTTPS is used. For example, when the attacker controls the device being used or when is able to trick users to install the self-signed certificates from the MitM tool being used.

Professionally I have wrote several blogs on the subject that I invite you to read:

Examples of when it can happen over HTTP

when a device connects to a wifi router.
compromised wifi router.
when the wifi router connects to the ISP using HTTP.
ISP sniffing due to law enforcement.
when the ISP connects to the destination several exchange points will be reached and state agents can be sniffing the traffic, like governments.

Examples on how they can happen:

session hijacking due to the use of unprotected cookies, like in Facebook and Twitter cases, by using a tool like Firesheep.
loading web forms hover HTTP, even when they post to HTTPS, because they may have been modified by a MitM attack. For example, they may have the post URL modified to target a malicious web server, instead of the one serving the website.
malicious/unauthorized tracking of our activity, by injecting things into the content we send/receive. Some ISPs do this.
injected malware in a compromised router/isp or any other exchange point. Like the Tunisia Keystroke logger inject on Facebook, Gmail and Yahoo login pages loaded hover HTTP.
DNS hijacking by compromising our DNS resolvers to serve the site from a server they control, making this way possible for them to change/replace the original content when the site is served hover HTTP. If the site is being served hover HTTPS, than to serve their own content they also need the valid TLS certificate for the site.
routers compromised - like here, here and here.

HTTPS Helps to mitigate:

phishing attacks - because when we click in a fake link to a website served hover HTTPS, they can’t spoof the TLS certificate for the original website.
malicious host files - if the host file in our device is compromised and points a HTTPS website to a different IP address, the TLS certificate will not match, despite the URL in the browser being the correct one.
Cross Site Request Forgery or CSRF - like on the routers compromised example given above.

TOC

HTTPS and Headers

HTTPS Strict Transport Security (HSTS)

This a header field we can set to instruct the browser to always make the initial and all subsequent requests to our website hover HTTPS.

For example:

# The max age value is in seconds.
Strict-Transport-Security: max-age=63072000; includeSubdomains; preload

The includeSubdomains policy means that all sub domains of the website will benefit of the same security policy.

The preload policy means that the browser should include it on the local preload list for the device, but not into the HSTS Preload list shipped with the browser on installation, and that is then kept up to date each time the browsers receives an update.

When the website is not in the HSTS Preload list but uses the preload policy and the user types example.com instead of https://example.com then the HSTS header to enforce the browser to always use HTTPS is fetched over an insecure HTTP connection, therefore subject to be manipulated in a MitM attack.

Having the backend server doing a redirect from HTTP to HTTPS is not enough to protect a user from having the request for example.com manipulated or redirected in a MitM attack.

So, to protect the website from being vulnerable to this scenario is recommended to add it into the hstspreload.org list, and all the browsers with the updated list will always make a request to https://example.com when the user just types example.com.

Browser support for the HSTS header can be checked at caniuse.com.

Content Security Policy (CSP)

The Content-Security-Policy is the name of an HTTP header used by modern browsers to improve security when loading a web page. It can also be used as a meta-tag in the web page HTML.

The CSP header tells to the browser what is allowed and not allowed to load and use, like Javascript, CSS, Iframes, inline styles, inline Javascript and much more.

To learn more about all the possible CSP policies I would recommend you to visit the content-security-policy.com website or visit the Mozilla Developer Docs.

To test the CSP header on your website you can use the securityheaders.com online tool, but bear in mind that this tool doesn’t read HTML meta tags.

Let’s see some brief examples of using a CSP policy.

Upgrade insecure requests

This CSP policy will instruct the browser to upgrade all request from HTTP to HTTPS in a web page.

When using the CSP header, the response to the request to fetch the web page must contain an header like:

Content-Security-Policy: upgrade-insecure-requests

When using the CSP meta tag, the web page HTML must contain something like this:

<html>
    <head>
        <meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests">
    </head>
</html>

This is useful to:

ensure third party content embedded in our page is not loaded over HTTP, like Facebook, Twitter or Disqus comments.
ensure that in case we forgot in our own code to load something over HTTPS it will be forced to use it.

Browser support can be checked on the caniuse.com website.

Block all mixed content

Header example:

Content-Security-Policy: block-all-mixed-content

HTML meta tag example:

<html>
    <head>
        <meta HTTP-equiv="Content-Security-Policy" content="block-all-mixed-content">
    </head>
</html>

Browser support can be checked at caniuse.com.

HTTP Referrer

When a user clicks in a HTTP link on a HTTPS page the referrer header will not be set in order to protect from potentially sending sensitive information in the HTTPS referrer link to the HTTP page, because if it was set then a MitM attack could easily obtain that sensitive information.

However the referrer header from an HTTPS page to another HTTPS page is always sent, meaning that sensitive information on it will be sent to the destination, therefore exposed/leaked to them.

So if the HTTPS destination page is not in our control we may not want the referrer header to be set and to control this we have some options:

Content Security Policy Referrer Header
The Referrer Policy Header
URL Mask Service - This will hide the referrer.
Shortening URL services should also work to hide the referrer, need to test it.

Support for the referrer policy header can be checked at caniuse.com.

Secure Cookies

Cookies must always be set with the Secure flag enabled to prevent them to be sent over HTTP, preventing them from being easily accessed by unintended parties, like in a MitM attack.

To prevent Javascript from accessing the cookies we also need to enable the httpOnly flag in the cookie.

We can read more about restricting access to cookies in this page from Mozilla developers.

TOC

Content Links

As rule of thumb Relative URLs should be only used for external links and for internal links we should always use Relative Paths.

Internal link is any link referencing anything under the same domain we served the content.

External link is any link referencing any link not belonging to the current domain, even a sub domain of the domain serving the content.

Given domain example.com as the one used to serve the content, than api.example.com is considered a external link, but example.com/api is an internal link.

Relative URLs

When we use external links in our content we normally use the absolute URL like http://external-link.com, but this is coupling us to a specific URL scheme, the HTTP one an when we implement HTTPS we will need to go back to our code and find all this URLs to fix them.

A best approach is to use a Relative URL where the URL scheme is omitted, therefore the browser prepends the URL scheme used to serve the content any time it needs to use this link.

So instead of using:

http://external-link.com we should use //external-link.com

Using //external-link.com will be interpreted by the browser as:

http://external-link.com when content is served over HTTP.
https://external-link.com when content is served over HTTPS.

Relative Paths

When serving the content with local assets or linking to internal resources is preferable to use the relative path instead of the relative URL.

Every time the browser finds a relative path it prepends the URL used to serve the content before trying to use it.

So instead of using:

//example.com/css/global.css we should use css/global.css
//example.com/users/1 we should use users/1
http://example.com/css/global.css we should use css/global.css
http://example.com/users/1 we should use users/1

Using css/global.css will be interpreted by the browser as:

http://example.com/css/global.css when content is served over HTTP.
https://example.com/css/global.css when content is served over HTTPS.

TOC

Brotli Compression

This is a new compression algorithm only available over HTTPS, that was introduced by Google in 2015.

This algorithm improves compression by around 25%.

A more in depth article by Scott Helme can be found here.

TOC

HTTPS Acronyms

Explaining the meaning of some acronyms used when talking about HTTPS implementations.

CAA - Certificate Authority Authorization

At the DNS level we can specify what Certificate Authorities we allow to issue certificates for the domain.

This also protects us from Certification Authorities that have been compromised, like the already mentioned DigiNotar Iran case.

CRL - Certificate Revocation List

This is a list that each Certification Authority needs to maintain to list all is revoked certificates, so that all clients using it can know what certificates they can’t trust any more.

So, this is a list that will be shipped with the browser or with the Operating System, therefore making the revocation process slow and dependent on our system updates.

DANE - DNS Based Authentication of Named Entities

Used to specify at DNS level the certificate keys being used. DNSSEC must be enabled to use DANE.

This helps to protect the compromise of a Certificate Authority, like the famous DigiNotar Iran case.

A brief explanation how they Certificate Authorities can be attacked can be found here.

DNSSEC - Domain Name System Security Extensions

Prevents the forgery of DNS records for the domain, guaranteeing this way the integrity of the Domain.

The forgery of DNS records allows an attacker to route the traffic to their own servers to what so ever they want with it.

HPKP - HTTP Public Key Pinning

Defines the certificate public keys that are admissible by the client for each domain.

In is attributes we can:

list all the public keys for allowed the certificate.
specify the max age.
tell if it applies to sub domains.
provide the URL to report violations for attempts on use of invalid public keys.

This is implemented in the response with the public-key-pins header.

A draw back of this is that we need to have a first trusted request, so that subsequent calls to the domain can be protected from man in the middle attacks.

OCSP - Online Certificate Status Protocol

An alternative to CRL where the Certificate Authorities implement the OCSP protocol to handle almost in real time the certificates revocation.

OCSP can cause some privacy issues, therefore OCSP Stapling is preferred.

In a OCSP implementation the website is responsible to validate the certificate by calling the OCSP server of the Certification Authority and return the result in the TLS hand shake between server and client.

PFS - Perfect Forward Secrecy

In case a private key for the certificate is compromise it protects any past sessions encrypted with that private key to become also compromised.

PKP - Public Key Pinning

The client will only accept certificates from a predefined list with the public keys of the certificates it can trust on.

This by itself does not guarantee to the client that a certificate in the predefined list is the correct one for the domain… Remember the DigiNotar Iran case.

SAN - Subject Alternative Name

Makes possible to use multiple domain names in the same certificate.

SNI - Server Name Indication

Enables multiple certificates in the same IP address.

SSL - Secure Sockets Layer

SSL was commonly used until 2014 Poodle attack to implement HTTPS but is now suppressed by TLS.

TLS - Transport Layer Security

Nowadays TLS is the standard to implement HTTPS, once is more secure and fast and with it the Poodle attack would havenot succeeded.

TOC

Tools

Automated Installation of Certificates - Automates the process of acquire/renew/install Let’s Encrypt certificates.
Mozilla Observatory - The most complete set of tools to test the website security.
SSL Server Test - The famous Qualys SSL Labs tester.
Security Headers Checker
Real Time Security Reporting
Fiddler - Proxy to debug HTTP and HTTPS.
BadSSL - site to show all that can go wrong with an HTTPS implementation.
HSTS Preload - verifies if a site is in the browsers HTTPS preload list, if they are eligible to be there or to submit a site to this preload list.
Chromium Preload List - List of all preloaded sites.
Browser Compatibility Checker for HSTS.
HTTP Vs HTTPS Speed - Compare speed of loading this website hover HTTP vs HTTPS connection.

TOC

Links

Free Certificates

Lets’s Encrypt - Let’s Encrypt gives us certificates for free with a 3 months expiring period.

Cheat Sheets

Scott Helme Cheat Sheet

Learning Resources

Blogs

Articles

Is TLS Fast Yet - website about speed impact when using secure communications.
DigiNotar CA Compromised - The history of how Iran compromised this CA to spy on is citizens email accounts.
Certificate Authorities Attacks - A brief explanation how they can get compromised.

TOC

Cover Photo by Pixabay from Pexels

Disclaimer: What I expressed here is only in my behalf and doesn't represent the company I work for, or any previous one, neither my family, friends, colleagues or anyone else unless I explicitly say so.