Install Brotli Compression on Nginx (ngx_brotli)

Brotli is the next generation lossless compression algorithm for web applications. I discovered this fantastic new compression algorithm while working with one of my customers.

It is the new RFC 7932 standart that provides a better solution than the traditional and beloved gzip. Nowadays, it is supported by 90% of the most popular browsers, so it is almost the definitive replacement for gzip compression.

Browser compatibility is pretty good in fact:

• Safari browser supports Brotli since version 11
• Edge browser from Microsoft supports Brotli since version 15
• Firefox supports Brotli  since v44
• Chrome started supporting Brotli since v49
• Opera supports Brotli since version 36

This compression mechanism uses a combination of LZ77 algorightm and Huffman code to increase the compression rate in around 25% more than using traditional gzip compression.

In terms of speed, it is almost the same as mod_deflate / mod_gzip, but it offers a more aggressive data compression, and also offers better decompression than current LZMA implementations.

The following video is a big proof of how brotli compression can help to speed up apps & at the same time reduce data transfered between hosts.

According to Akamai tests against the top 1000 websites on the planet, Brotli shows its power in numbers:

• Offers 14% more compression for Javascript files than using Gzip.
• Uses 17% less data than gzip for CSS files
• Shows 21% improvement in compression than mod_gzip for HTML static files.

ngx_brotli is the Nginx module in charge of  the compression task, which is a set of two modules:

• ngx_brotli filter module – used to compress responses on-the-fly,
• ngx_brotli static module – used to serve pre-compressed files.

The best place to test brotli is using Nginx, my beloved web server. And that’s where we are headed to, let’s try to install and configure Brotli on Nginx using ngx_brotli module.

Installing brotli ngx_brotli on Ubuntu 16.04

Ubuntu usually comes with old Nginx official versions, that of course do not include the new Brotli compression module.

However, using a 3rd party repo that provides Nginx HDA Bundle (Dynamic Modules Power) will make our life easier while adding Brotli compression support.

Run this commands:

sudo apt-add-repository -y ppa:hda-me/nginx-stable
sudo apt-get update
sudo apt-get install brotli nginx nginx-module-brotli

Now let’s try to see if Brotli was indeed compiled into Nginx:

nginx -V 2>&1 | tr ' ' '\n' | grep brotli

If you see this output then it was installed ok

--add-dynamic-module=debian/extra/ngx_brotli

Restart Nginx and start enjoying Brotli using Nginx on Ubuntu.

sudo systemctl restart nginx

Install Brotli on RHEL / CentOS 7.x using RPM

For RedHat based distros like CentOS and RHEL, you can also use a non official Nginx RPM repo that already comes with Brotli support enabled by default.

cd /etc/yum.repos.d
wget https://repo.codeit.guru/codeit.mainline.el`rpm -q --qf "%{VERSION}" $(rpm -q --whatprovides redhat-release)`.repo
yum install nginx

Now check out to see if Nginx was installed with the built in Brotli compression support:

nginx -V 2>&1 | tr ' '  '\n' | grep brotli

This should be the expected output:

--add-module=/tmp/nginx/ngx_brotli

Restart Nginx to apply changes and start using Brotli.

systemctl restart nginx

Installing Brotli from source

For both distros, Ubuntu and RHEL/CentOS, you can also choose to compile your own Nginx server with Brotli support enabled.

But before we need to make sure we have all the required packages for the manual compilation and post install process.

RHEL and CentOS users:

yum groupinstall 'Development Tools' -y

Ubuntu and Debian:

sudo apt install build-essential -y

Download Nginx source files

Move to Nginx’s download area and then download the latest Nginx stable version, then run this commands (make sure you replace the Nginx version with the latest seen on the Nginx website):

mkdir /root/temp && cd /root/temp
wget http://nginx.org/download/nginx-1.13.6.tar.gz
tar -xzvf nginx-1.13.6.tar.gz

Download Brotli Nginx module (ngx_brotli) from the official repo at Github

cd /root/temp
git clone https://github.com/google/ngx_brotli.git
cd ngx_brotli
git submodule update --init --recursive

The output should be something like:

[[email protected]:~]git submodule update --init --recursive
Submodule 'deps/brotli' (https://github.com/google/brotli.git) registered for path 'deps/brotli'
Cloning into 'deps/brotli'...
remote: Counting objects: 4980, done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 4980 (delta 25), reused 52 (delta 19), pack-reused 4895
Receiving objects: 100% (4980/4980), 30.07 MiB | 10.98 MiB/s, done.
Resolving deltas: 100% (3121/3121), done.
Submodule path 'deps/brotli': checked out '222564a95d9ab58865a096b8d9f7324ea5f2e03e'
Submodule 'research/esaxx' (https://github.com/hillbig/esaxx) registered for path 'research/esaxx'
Submodule 'terryfy' (https://github.com/MacPython/terryfy.git) registered for path 'terryfy'
Cloning into 'research/esaxx'...
remote: Counting objects: 32, done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 32 (delta 10), reused 32 (delta 10), pack-reused 0
Unpacking objects: 100% (32/32), done.
Submodule path 'deps/brotli/research/esaxx': checked out 'ca7cb332011ec37a8436487f210f396b84bd8273'
Cloning into 'terryfy'...
remote: Counting objects: 918, done.
remote: Total 918 (delta 0), reused 0 (delta 0), pack-reused 918
Receiving objects: 100% (918/918), 165.17 KiB | 0 bytes/s, done.
Resolving deltas: 100% (478/478), done.
Submodule path 'deps/brotli/terryfy': checked out '8bb673f4410819df06920fdcfd24e18d235d84f7'
[[email protected]:~]

Get your existing Nginx configuration arguments

Run -V parameter to grab your existing Nginx configuration arguments to keep everything in the same state as now:

nginx -V

You will get your current Nginx configuration, as you see on this example:

[[email protected]:~]nginx -V
nginx version: nginx/1.13.6
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) 
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

Grab everything after “configure arguments:” part, and copy that into your clipboard.

Now let’s add “./configure” at the beginning of a new line, after that pase the content of your “configure arguments”, and at the end add the ngx_brotli module, as you see below:

--add-module=/root/temp/ngx_brotli

The full ./configure command would look like:

./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'--add-module=/root/temp/ngx_brotli

Install the compiled Nginx package

The last step to install Nginx with Brotli is to run make & make install commands, as you see below:

make
make install

Now restart Nginx

service nginx restart

or

sudo systemctl restart nginx

Configuring Brotli compression in Nginx

Nginx is now installed with Brotli support, but it is not fully activated / enabled yet. Let’s enable brotli compression inside Nginx configuration files (nginx.conf).

Make sure you place the brotli directives as you see below, inside a http block.

http {
     ...
     ...
     brotli on;
     brotli_static on;
     brotli_types *;
     ...
     ...
}

After that, let’s review the Nginx configuration to see if it looks ok from a configuration syntax point of view:

nginx -t

Output should be something like:

[[email protected]:~]nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Explaining Brotli Directives

The configuration we used here included three Brotli compression directives.

• brotli = is used to activate brotli support for on-the-fly compression for incomingo requests.
• brotli_static = used to serve static pre-compressed files like css, js, etc with the .br extension at the end eg: jquery.js.br.
• brotli_types = used to specity specific MIME-types to be used, or use a wildcard, as we did on this example.

Other Brotli (ngx_brotli) directives

Apart from the ones mentioned before, there are many other brotli nginx directives that can be used to configure your compression. For example:

• brotli_comp_level =  sets brotli compression levels between 0 to 11, default level is 6.
• brotli_window = used to set Brotli windows size. Values are: 1k, 2k, 4k, 8k, 16k, 32k, 64k, 128k, 256k, 512k, 1m, 2m, 4m, 8m and 16m
• brotli_buffers = lets you specify size and number of buffers used during the compression process, default value is 32 4k|16 8k.
• brotli_min_length = configure a minimum length in order to have the requst compressed, determined by the Content-Length field in the HTTP headers.

Testing Brotli

How can I know if Brotli is working on my website?

The easiest way to check out if Brotli is using our Brotli Test, which will help you to identify if the Content-Encoding: br is present on the headers.