Pushing builds with butler

The only command that you need to remember isbutler push

butler push directory user/game:channel

Where:

• directory is what you want to upload. It can also be a .zip file.
• user/game is the project you're uploading
  • for example:finji/overland forhttps://finji.itch.io/overland — all lower-case
• channel is which slot you're uploading it to
  • for example:windows-beta,osx-bonus,linux-universal, orsoundtrack

Channel names will determine the initial set of tags of a slot, but you can always fix them later.

Okay, show me

If you push the same directory twice:

• The first push will take some time, all data actually needs to be uploaded

• The next pushes will be almost instantaneous, since nothing changed

These are two extreme cases — most of the time, you'll be somewhere in the middle,with maybe 5% to 20% fresh data in a push, saving 80% to 95% (more with compression)of your data/time/internet bandwidth.

Note: the first push is apatch too (fromthe empty container to the first versionof your game). It has savings as well, because butler uploads compressed data: you could be pushing a 300MB build but only sending 120MB over the network.

Pushing to a new channel will make a corresponding file appear on your game's page:

Pushing to the same channel again will update that file, once the build is processed.

Limits: currently, the itch.io backend will reject builds with a total uncompressedsize that exceeds 30GB.

Channel names

The name of a channel has meaning:

• If it containswin orwindows, it'll be tagged as a Windows executable
• If it containslinux, it'll be tagged as a Linux executable
• If it containsmac orosx, it'll be tagged as a Mac executable
• If it containsandroid, it'll be tagged as an Android application
• Channel names may contain multiple platforms (for example, a Java game could be pushedtowin-linux-mac-stable)

Limits: there is no way to tag channels with architecture (32-bit vs 64-bit) yet.

The general convention for channel names is: lower-case characters, separatedby dashes (aka.kebab-case).

Naming your channel correctly will ensure your build is probably taggedfrom the get-go. However, you can always change it later from theEdit game page,by using the checkboxes.

Don't forget toSave the game's page to apply your changes.

Clicking on the green channel button will show a list of builds and theircurrent status.

HTML / Playable in browser games

Tagging a channel as 'HTML5 / Playable in browser' needs to be done from theitch.io Edit game page, once the first build is pushed.

The page also needs to be set to 'HTML' rather than the default 'Downloadable'.

Specifying your own version number

By default, the itch.io backend will generate a constantly-increasing integerversion number for your builds, for each channel.

However, if you're already generating your own build number, you shouldpass it to butler with the--userversion option:

butler push mygame user/mygame:win32-final --userversion 1.1.0

You can also instruct butler to read the version number from a given file,if it fits your workflow better:

butler push mygame user/mygame:win32-final --userversion-file buildnumber.txt

Thebuildnumber.txt file should contain a single line with theversion or build number, in UTF-8 without BOM.

User-provided version numbers don't have any particular format -the ordering itch.io uses is the one builds are uploaded in.

Looking for updates

Players who prefer downloading directly rather than usingthe itch appdon't get automatic updates. You can use the following API endpoint to query the latest versionof a game and notify your players from within the game, that a new version is available:

GET https://itch.io/api/1/x/wharf/latest

Parameters:

• channel_name: the name of the channel to query
• either of these:
  • game_id: (numeric) identifier of the game (find it inEdit game)
  • target: user/game, just like thebutler push command

Response: a JSON object with the latest user-version for the given channel:

{  latest:"106"}

If the latest build doesn't have a user-version, thelatest field will not be present.

Example requests:

https://itch.io/api/1/x/wharf/latest?target=user/game&channel_name=win32-beta

or

https://itch.io/api/1/x/wharf/latest?game_id=123&channel_name=osx-final

(You can find thegame_id from theEdit game page's address)

Note: if the game's visibility level is set toPrivate, this endpoint will returnthe error 'invalid game', to avoid potentially leaking information about unreleased games.

Appendix A: Understanding the progress bar

butler push does a lot of work, most of it in parallel:

• Signature of the previous build (if any) is downloaded
• Files from the local directory/archive are scanned
• Differences between the local (newer) & remote (older) build are
  • computed (by comparing hashes, see theWharf specification)
  • ...and compressed (with a fast preset)
  • ...and uploaded toitch.io

In addition:

• butler doesn't use any additional disk space while uploading
• butler doesn't require a cache in-between uploads
  • You can upload from a different machine every time
• butler tries really hard to use less than 256MB of RAM

By design, progress bar shown bybutler push has two parts:

• The half-tone bar shows the progress of scanning files & generating the patch
• The full bar shows the progress of uploading the generated patch

Neither of these bars jump back. The half-tone bar may pause, waiting for thefull bar bar to catch up: this means butler is waiting to send the patch datait has already generated before scanning more files, and it helps keepingmemory usage reasonable.

The estimated time remaining is to be taken with a grain of salt, as areall ETAs.

Appendix B: Beeps 4 life

The default windows command-line uses a character set namedCode page 437. Its historical relevance goes back to the days where Unicode wasn't yet ubiquitous.¹

In the early days of butler, we were using the CP437 character007 to displaybullet points (whenever butler starts a new task). However, that particularcharacter is also used to emit a system beep².

This was a bug, of course, butearly users got attachedto it. It wassoothing. The bug has since been fixed, but the historical behavior can berestored by using the--beeps4life option.

Note that this option is not listed in the program's inline help.

Appendix C: Ignoring files

The recommended way to use butler push is to pass it a folder that isexactly yourrelease build, with no extra files. For example, if your build process generates*.pdb files, and you don't want to push that - create a copy of the build folderwithout them, or just remove them from the build folder before pushing.

However, because it is hard to resist feature requests, butler supports the--ignore flag, which lets you push a folderwithout some files.

Here are the rules:

• The pattern syntax is described inhttps://godoc.org/path/filepath#Match
• Default ignore patterns arelisted here
• You can use--ignore multiple times, once per pattern

For example, if you wanted to ignore all.pdb and.dSYM files, you woulddo something like:

butler push --ignore '*.pdb' --ignore '*.dSYM' my-build/ foo/bar:baz

Note: It's--ignore, with two dashes, not-ignore. Long flags alwaysuse two dashes.

To test your ignore patterns, ie. preview what would get pushed without pushing it,you can use the--dry-run flag. It shows a complete list of files that wouldget pushed, as well as a summary.

Example output without --ignore:

(snip)-rw-rw-rw- 895.69 KiB ucrtbase.dll-rw-rw-rw-  74.41 KiB ui_resources_200_percent.pak-rw-rw-rw-  81.82 KiB vcruntime140.dll-rw-rw-rw-        6 B version-rw-rw-rw-  56.10 KiB views_resources_200_percent.pak√ Would push 103.58 MiB (118 files, 2 dirs, 0 symlinks)

Example output with--ignore '*.dll':

(snip)-rw-rw-rw- 586.13 KiB resources/app.asar-rw-rw-rw- 233.71 KiB resources/electron.asar-rw-rw-rw-   1.12 MiB snapshot_blob.bin-rw-rw-rw-  74.41 KiB ui_resources_200_percent.pak-rw-rw-rw-        6 B version-rw-rw-rw-  56.10 KiB views_resources_200_percent.pak√ Would push 80.05 MiB (70 files, 2 dirs, 0 symlinks)

Appendix D: Dereferencing symlinks

As mentioned in Appendix C, we really really recommend that the folderyou push is the final version of your build folder, with no changes.

However, because saying no to feature requests requires more strengththan it does implementing said features sometimes, there is a way topush a folder with symlinks in it as if those symlinks were copiesof the folder/files they refer to.

So let's say you have:

- game-linux  - game-binary.x64  - data -> ../game-data- game-data  - level1.data

Then you could do:

butler push --dereference game-linux/ foo/bar:baz

Note thatthis will cause problems if you don't know what you're doing(and sometimes even if you know what you're doing). For example:

• Linux builds that ship with dynamic libraries tend to havesymlinks linkslibfoo.so => libfoo.so.3 andlibfoo.so.3 =>libfoo.so.3.14
• macOS builds in the form of app bundles (theonly macOS buildsyou should ever push) have lots of symlinks too, typically in theFrameworks/directory.

It's not that the builds you push won't work, it's that they'll includeseveral copies of the same file, and thus be larger than needed.

Appendix E: Pushing only if changed

Most times, you want to push unconditionally - even if nothing changed, justso that there are consistent version numbers across, say, Windows, macOS & Linux.

But sometimes, you might be pushing to a bunch of project pages / channels, andyou want to avoid generate empty/no-op patches. The--if-changed flag lets younot push anything if the contents on disk are exactly the same as the latest build.

To give you an example use case, this is used when updatinghttps://itchio.itch.io/itch-redists. Each upload on that page is a filein a git repository, and they're all deployed in the same CI pipeline. Typicallyonly one or two channels actually get changed, and--if-changed reduces patchingnoise.

¹. It still isn't really, but you get the idea. ↩

². Historically, from your computer'sPC speaker. Now, probably whatever sound Microsoft bundles with your version of Windows. ↩