Repository files navigation

It is a Flask extension that will help make your static files production readywith very minimal effort on your part. It does this by creating md5 taggedversions and gzip and / or brotli compressed versions of your static files byrunning aflask digest compile command that this extension adds to your Flaskapp.

It should be the last thing you do to your static files before uploading themto your server or CDN. Speaking of which, if you're using a CDN this extensionoptionally lets you configure a host URL that will get prepended to your staticfile paths. If you're not using a CDN, no problem everything will work as youwould expect by default.

Other web frameworks like Django, Ruby on Rails and Phoenix all have thisfeature built into their framework, and now with this extension Flask does too.

This extension will work if you're not using any asset build tools but at thesame time it also works with esbuild, Webpack, Grunt, Gulp or any other buildtool you can think of. This tool does not depend on or compete with existingasset build tools.

If you're already using Webpack or a similar tool, that's great. Webpack takescare of bundling your assets and helps convert things like SASS to CSS and ES6+JS to browser compatible JS. That is solving a completely different problemthan what this extension solves. This extension will further optimize yourstatic files after your build tool produces its output files.

This extension does things that Webpack alone cannot do because in order forthings like md5 tagging to work Flask needs to be aware of how to map thosehashed file names back to regular file names you would reference in your Jinja2 templates.

There's 3 pieces to this extension:

1. It adds a custom Flask CLI command to your project. When you run thiscommand it looks at your static files and then generates an md5 taggedversion of each file along with optionally compressing them with gzipand / or brotli.

2. When the above command finishes it creates acache_manifest.json file inyour static folder which maps the regular file names, such asimages/flask.png toimages/flask-f86b271a51b3cfad5faa9299dacd987f.png.

3. It adds a new template helper calledstatic_url_for which uses Flask'surl_for under the hood but is aware of thecache_manifest.json file soit knows how to resolveimages/flask.png to the md5 tagged file name.

This 25 minute video goes over using this extension but it also spends a lotof time on the "why" where we cover topics like cache busting and why IMO youmight want to use this extension in all of your Flask projects.

If you prefer reading instead of video, this README file covers installing,configuring and using this extension too.

• FLASK_STATIC_DIGEST_HOST_URL has been added to configure an optional external host, aka. CDN (explained here)
• If your blueprints have static files they will get digested now too (including nested blueprints!)
• Optional Brotli support has been added
• FLASK_STATIC_DIGEST_COMPRESSION has been added to control compression (explained here)

• Installation
• Using the newly added Flask CLI command
• Going over the Flask CLI commands
• Configuring this extension
• Modifying your templates to use static_url_for instead of url_for
• Potentially updating your .gitignore file
• FAQ
  • What about development vs production and performance implications?
  • Why bother compressing your static files here instead of with nginx?
  • How do you use this extension with Webpack or another build tool?
  • Migrating from Flask-Webpack
  • How do you use this extension with Docker?
  • How do you use this extension with Heroku?
  • What about user uploaded files?
• About the author

You'll need to be running Python 3.6+ and using Flask 1.0 or greater.

pip install Flask-Static-Digest

To install with Brotli support:

pip install Flask-Static-Digest[brotli]

├── hello│   ├── __init__.py│   ├── app.py│   └── static│       └── css│           ├── app.css└── requirements.txt

fromflaskimportFlaskfromflask_static_digestimportFlaskStaticDigestflask_static_digest=FlaskStaticDigest()defcreate_app():app=Flask(__name__)flask_static_digest.init_app(app)@app.route("/")defindex():return"Hello, World!"returnapp

A more complete example app can be found in thetests/directory.

You'll want to make sure to at least set theFLASK_APP environment variable:

export FLASK_APP=hello.appexport FLASK_ENV=development

Then run theflask binary to see its help menu:

Usage: flask [OPTIONS] COMMAND [ARGS]...  ...Options:  --version  Show the flask version  --help     Show this message and exit.Commands:  digest  md5 tag and compress static files.  routes  Show the routesfor the app.  run     Run a development server.  shell   Run a shellin the app context.

If all went as planned you should see the newdigest command added to thelist of commands.

Runningflask digest will produce this help menu:

Usage: flask digest [OPTIONS] COMMAND [ARGS]...  md5 tag and compress static files.Options:  --help  Show this message and exit.Commands:  clean    Remove generated static files and cache manifest.  compile  Generate optimized static files and a cache manifest.

Each command is labeled, but here's a bit more information on what they do.

Inspects your Flask app's and blueprint'sstatic_folder and uses that as boththe input and output path of where to look for and create the newly digestedand compressed files.

At a high level it recursively loops over all of the files it finds in thatdirectory and then generates the md5 tagged and compressed versions of eachfile. It also creates acache_manifest.json file in the root of yourstatic_folder.

That manifest file is machine generated meaning you should not edit it unlessyou really know what you're doing.

This file maps the human readable file name of let's sayimages/flask.png tothe digested file name. It's a simple key / value set up. It's basically aPython dictionary in JSON format.

In the end it means if your static folder looked like this originally:

• css/app.css
• js/app.js
• images/flask.png

And you decided to run the compile command, it would now look like this:

• css/app.css
• css/app.css.gz
• css/app-5d41402abc4b2a76b9719d911017c592.css
• css/app-5d41402abc4b2a76b9719d911017c592.css.gz
• js/app.js
• js/app.js.gz
• js/app-098f6bcd4621d373cade4e832627b4f6.js
• js/app-098f6bcd4621d373cade4e832627b4f6.js.gz
• images/flask.png
• images/flask.png.gz
• images/flask-f86b271a51b3cfad5faa9299dacd987f.png
• images/flask-f86b271a51b3cfad5faa9299dacd987f.png.gz
• cache_manifest.json

Your md5 hashes will be different because it depends on what the contents ofthe file are.

Inspects your Flask app's and blueprint'sstatic_folder and uses that as theinput path of where to look for digested and compressed files.

It will recursively delete files that have a file extension of.gz or.brand files that have been digested. It determines if a file has been digestedbased on its file name. In other words, it will delete files that match thisregexpr"-[a-f\d]{32}".

In the end that means if you had these 6 files in your static folder:

• images/flask.png
• images/flask.png.gz
• images/flask.png.br
• images/flask-f86b271a51b3cfad5faa9299dacd987f.png
• images/flask-f86b271a51b3cfad5faa9299dacd987f.png.gz
• images/flask-f86b271a51b3cfad5faa9299dacd987f.png.br

And you decided to run the clean command, the last 5 files would be deletedleaving you with the originalimages/flask.png.

By default this extension will create md5 tagged versions of all files it findsin your configuredstatic_folder. It will also create gzip'ed versions of eachfile and it won't prefix your static files with an external host.

If you don't like any of this behavior or you wish to enable brotli you canoptionally configure:

FLASK_STATIC_DIGEST_BLACKLIST_FILTER= []# If you want specific extensions to not get md5 tagged you can add them to# the list, such as: [".htm", ".html", ".txt"]. Make sure to include the ".".FLASK_STATIC_DIGEST_COMPRESSION= ["gzip"]# Optionally compress your static files, supported values are:#   []                 avoids any compression#   ["gzip"]           uses gzip#   ["brotli"]         uses brotli (prefer either gzip or both)#   ["gzip", "brotli"] uses bothFLASK_STATIC_DIGEST_HOST_URL=None# When set to a value such as https://cdn.example.com and you use static_url_for# it will prefix your static path with this URL. This would be useful if you# host your files from a CDN. Make sure to include the protocol (aka. https://).

You can override these defaults in your Flask app's config file.

We're all familiar with this code right?

<imgsrc="{{ url_for('static', filename='images/flask.png') }}"width="480"height="188"alt="Flask logo"/>

When you put the above code into a Flask powered Jinja 2 template, it turnsinto this:

<imgsrc="images/flask.png"width="480"height="188"alt="Flask logo"/>

The path might vary depending on how you configured your Flask app'sstatic_folder but you get the idea.

Let's use the same example as above:

<imgsrc="{{ static_url_for('static', filename='images/flask.png') }}"width="480"height="188"alt="Flask logo"/>

But now take a look at the output this produces:

<imgsrc="/images/flask-f86b271a51b3cfad5faa9299dacd987f.png"width="480"height="188"alt="Flask logo"/>

Or if you setFLASK_STATIC_DIGEST_HOST_URL = "https://cdn.example.com" itwould produce:

<imgsrc="https://cdn.example.com/images/flask-f86b271a51b3cfad5faa9299dacd987f.png"width="480"height="188"alt="Flask logo"/>

Instead of usingurl_for you would usestatic_url_for. This uses Flask'surl_for under the hood so things like_external=True and everything elseurl_for supports is available to use withstatic_url_for.

That means to use this extension you don't have to do anything other thaninstall it, optionally run the CLI command to generate the manifest and thenrename your static file references to usestatic_url_for instead ofurl_for.

If your editor supports performing a find / replace across multiple files youcan quickly make the change by findingurl_for('static' and replacing thatwithstatic_url_for('static'. If you happen to use double quotes instead ofsingle quotes you'll want to adjust for that too.

If you're using something like Webpack then chances are you're already gitignoring the static files it produces as output. It's a common pattern tocommit your Webpack source static files but ignore the compiled static filesit produces.

But if you're not using Webpack or another asset build tool then the staticfiles that are a part of your project might have the same source anddestination directory. If that's the case, chances are you'll want to gitignore the md5 tagged files as well as the compressed andcache_manifest.jsonfiles from version control.

For clarity, you want to ignore them because you'll be generating them on yourserver at deploy time or within a Docker image if you're using Docker. Theydon't need to be tracked in version control.

Add this to your.gitignore file to ignore certain files this extensioncreates:

*-[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f].**.gzcache_manifest.json

This allows your original static files but ignores everything else thisextension creates. I am aware at how ridiculous that ignore rule is for the md5hash but using[0-9a-f]{32} does not work. If you know of a better way,please open a PR!

You would typically only run the CLI command to prepare your static files forproduction. Runningflask digest compile would become a part of your buildprocess -- typically after you pip install your dependencies.

In development when thecache_manifest.json likely doesn't existstatic_url_for callsurl_for directly. This allows thestatic_url_forhelper to work in both development and production without any fuss.

It's also worth pointing out the CLI command is expected to be run before youeven start your Flask server (or gunicorn / etc.), so there's no perceivablerun time performance hit. It only involves doing 1 extra dictionary lookup atrun time which is many orders of magnitude faster than even the most simpledatabase query.

In other words, this extension is not going to negatively impact theperformance of your web application. If anything it's going to speed it up andsave you money on hosting.

That's because compressed files can be upwards of 5-10x smaller so there's lessbytes to transfer over the network.

Also with md5 tagging each file it means you can configure your web server suchas nginx to cache each file forever. That means if a user visits your site asecond time in the future, nginx will be smart enough to load it from theirlocal browser's cache without even contacting your server. It's a 100% locallook up.

This is as efficient as it gets. You can't do this normally without md5 taggingeach file because if the file changes in the future, nginx will continueserving the old file until the cache expires so users will never see yourupdates. But due to how md5 hashing works, if the contents of a file changes itwill get generated with a new name and nginx will serve the uncached new file.

This tactic is commonly referred to as "cache busting" and it's a very goodidea to do this in production. You can even go 1 step further and serve yourstatic files using a CDN. Using this cache busting strategy makes configuringyour CDN a piece of cake since you don't need to worry about ever expiring yourcache manually.

You would still be using nginx's gzip / brotli features, but now instead ofnginx having to compress your files on the fly at run time you can configurenginx to use the pre-made compressed files that this extension creates.

This way you can benefit from having maximum compression without having nginxwaste precious CPU cycles compressing files on the fly. This gives you the bestof both worlds -- the highest compression ratio with no noticeable run timeperformance penalty.

It works out of the box with no extra configuration or plugins needed forWebpack or your build tool of choice.

Typically the Webpack (or another build tool) work flow would look like this:

• You configure Webpack with your source static files directory
• You configure Webpack with your destination static files directory
• Webpack processes your files in the source directory and copies them to thedestination directory
• Flask is configured to serve static files from that destination directory

For example, your source directory might beassets/ inside of your projectand the destination might bemyapp/static.

This extension will look at your Flask configuration for thestatic_folderand determine it's set tomyapp/static so it will md5 tag and compress thosefiles. Your Webpack source files will not get digested and compressed.

Flask-Webpack is another extension Iwrote a long time ago which was specific to Webpack but had a similar idea tothis extension. Flask-Webpack is now deprecated in favor ofFlask-Static-Digest. Migrating is fairly painless. There are a number ofchanges but on the bright side you get to delete more code than you add!

• RemoveFlask-Webpack fromrequirements.txt
• Remove all references to Flask-Webpack from your Flask app and config
• Removemanifest-revision-webpack-plugin frompackage.json
• Remove all references to this webpack plugin from your webpack config
• AddFlask-Static-Digest torequirements.txt
• Add the Flask-Static-Digest extension to your Flask app

• Replacestylesheet_tag('main_css') | safe withstatic_url_for('static', filename='css/main.css')
• Replacejavascript_tag('main_js') | safe withstatic_url_for('static', filename='js/main.js')
• Replace any occurrences ofasset_url_for('foo.png') withstatic_url_for('static', filename='images/foo.png')

It's really no different than without Docker, but instead of runningflask digest compile on your server directly at deploy time you would run it insideof your Docker image at build time. This way your static files are already setup and ready to go by the time you pull and use your Docker image inproduction.

You can see a fully working example of this in the open source version of myBuild a SAAS App withFlask course. Itleverages Docker's build arguments to only compile the static files whenFLASK_ENV is set toproduction. The key files to look at are theDockerfile,docker-compose.yml and.env files. That wires up the buildarguments and env variables to make it work.

If you're deploying to Heroku using the Python buildpack you can follow these 2 steps:

1. Create abin/post_compile file in your project's source code
2. Copy the lines below into thebin/post_compile file, save it and commit the changes

#!/usr/bin/env bashset -eecho"-----> Digesting static files"cd"${1}"&& flask digest compile

The next time you push your code this script will run after your pipdependencies are installed. It will run before your slug is compiled whichensures that the digested files are available before any traffic is served toyour Dyno.

You can view how this file gets executed by Heroku in theirPython buildpack'ssourcecode.

Let's say that besides having static files like your logo and CSS / JavaScriptbundles you also have files uploaded by users. This could be things like a useravatar, blog post images or anything else.

You would still want to md5 tag and compress these files but now we've runinto a situation. Theflask digest compile command is meant to be run atdeploy time and it could potentially be run from your dev box, inside of aDocker image, on a CI server or your production server. In these cases youwouldn't have access to the user uploaded files.

But at the same time you have users uploading files at run time. They arechanging all the time.

Needless to say you can't use theflask digest compile command to digestuser uploaded files. Thecache_manifest.json file should be reserved forfiles that exist in your code repo (such as your CSS / JS bundles, maybe alogo, fonts, etc.).

The above files do not change at run time and align well with running theflask digest compile command at deploy time.

For user uploaded content you wouldn't ever write these entries to the manifestJSON file. Instead, you would typically upload your files to disk, S3 orsomewhere else and then save the file name of the file you uploaded into yourlocal database.

So now when you reference a user uploaded file (let's say an avatar), you wouldloop over your users from the database and reference the file name from the DB.

There's no need for a manifest file to store the user uploaded files becausethe database has a reference to the real name and then you are dynamicallyreferencing that in your template helper (static_url_for), so it's never ahard coded thing that changes at the template level.

What's cool about this is you already did the database query to retrieve therecord(s) from the database, so there's no extra database work to do. All youhave to do is reference the file name field that's a part of your model.

But that doesn't fully solve the problem. You'll still want to md5 tag andcompress your user uploaded content at run time and you would want to do thisbefore you save the uploaded file into its final destination (local file system,S3, etc.).

This can be done completely separate from this extension and it's really goingto vary depending on where you host your user uploaded content. For example someCDNs will automatically create compressed files for you and they use things likean ETag header in the response to include a unique file name (and this is whatyou can store in your DB).

So maybe md5 hashing and maybe compressing your user uploaded content becomes anapp specific responsibility, although I'm not opposed to maybe creating helperfunctions you can use but that would need to be thought out carefully.

However the implementation is not bad. It's really only about 5 lines of codeto do both things. Feel free toCTRL + F around thecodebaseforhashlib,gzip andbrotli and you'll find the related code.

So with that said, here's a work flow you can do to deal with this today:

• User uploads file
• Your Flask app potentially md5 tags / gzips the file if necessary
• Your Flask app saves the file name + compressed file to its final destination (local file system, S3, etc.)
• Your Flask app saves the final unique file name to your database

That final unique file name would be the md5 tagged version of the file thatyou created or the unique file name that your CDN returned back to you. I hopethat clears up how to deal with user uploaded files and efficiently servingthem!

• Nick Janetakis |https://nickjanetakis.com |@nickjanetakis

If you're interested in learning Flask I have a 17+ hour video course calledBuild a SAAS App withFlask.It's a course where we build a real world SAAS app. Everything about the courseand demo videos of what we build is on the site linked above.