Repository files navigation

• THE PROJECT HAS VERY LIMITED SUPPORT
• NO NEW FEATURES ARE IN THE WORKS
• MOST BUGS WILL GO UNFIXED

gitstatus is a 10x faster alternative togit status andgit describe. Its primary usecase is to enable fast git prompt in interactive shells.

Heavy lifting is done bygitstatusd -- a custom binary written in C++. It comes with Zsh andBash bindings for integration with shell.

1. Using from Zsh
2. Using from Bash
3. Using from other shells
4. How it works
5. Benchmarks
6. Why fast
7. Requirements
8. Compiling
9. License

The easiest way to take advantage of gitstatus from Zsh is to use a theme that's already integratedwith it. For example,Powerlevel10k is a flexible andfast theme with first-class gitstatus integration. If you install Powerlevel10k, you don't need toinstall gitstatus.

For those who wish to use gitstatus without a theme, there isgitstatus.prompt.zsh. Install it as follows:

git clone --depth=1 https://github.com/romkatv/gitstatus.git~/gitstatusecho'source ~/gitstatus/gitstatus.prompt.zsh'>>!~/.zshrc

Users in China can use the official mirror on gitee.com for faster download.
中国大陆用户可以使用 gitee.com 上的官方镜像加速下载.

git clone --depth=1 https://gitee.com/romkatv/gitstatus.git~/gitstatusecho'source ~/gitstatus/gitstatus.prompt.zsh'>>!~/.zshrc

Alternatively, if you have Homebrew installed:

brew install romkatv/gitstatus/gitstatusecho"source$(brew --prefix)/opt/gitstatus/gitstatus.prompt.zsh">>!~/.zshrc

(If you choose this option, replace~/gitstatus with$(brew --prefix)/opt/gitstatus/gitstatusin all code snippets below.)

Make sure to disable your current theme if you have one.

This will give you a basic yet functional prompt with git status in it. It'sover 10x faster than any alternative that can give you comparable prompt. In orderto customize it, setPROMPT and/orRPROMPT at the end of~/.zshrc after sourcinggitstatus.prompt.zsh. Insert${GITSTATUS_PROMPT} where you want git status to go. For example:

source~/gitstatus/gitstatus.prompt.zshPROMPT='%~%#'# left prompt: directory followed by %/# (normal/root)RPROMPT='$GITSTATUS_PROMPT'# right prompt: git status

The expansion of${GITSTATUS_PROMPT} can contain the following bits:

$GITSTATUS_PROMPT_LEN tells you how long$GITSTATUS_PROMPT is when printed to the console.gitstatus.prompt.zsh has an example of using it to truncate the currentdirectory.

If you'd like to change the format of git status, or want to have greater control over theprocess of assemblingPROMPT, you can copy and modify parts ofgitstatus.prompt.zsh instead of sourcing the script. Your~/.zshrcmight look something like this:

source~/gitstatus/gitstatus.plugin.zshfunctionmy_set_prompt() {  PROMPT='%~%#'  RPROMPT=''if gitstatus_query MY&& [[$VCS_STATUS_RESULT== ok-sync ]];then    RPROMPT=${${VCS_STATUS_LOCAL_BRANCH:-@${VCS_STATUS_COMMIT}}//\%/%%}# escape %(( VCS_STATUS_NUM_STAGED))&& RPROMPT+='+'(( VCS_STATUS_NUM_UNSTAGED))&& RPROMPT+='!'(( VCS_STATUS_NUM_UNTRACKED))&& RPROMPT+='?'fi  setopt no_prompt_{bang,subst} prompt_percent# enable/disable correct prompt expansions}gitstatus_stop'MY'&& gitstatus_start -s -1 -u -1 -c -1 -d -1'MY'autoload -Uz add-zsh-hookadd-zsh-hook precmd my_set_prompt

This snippet is sourcinggitstatus.plugin.zsh rather thangitstatus.prompt.zsh. The formerdefines low-level bindings that communicate with gitstatusd over pipes. The latter is a simplescript that uses these bindings to assemble git prompt.

UnlikePowerlevel10k, code based ongitstatus.prompt.zsh is communicating with gitstatusd synchronously. Thiscan make your prompt slow when working in a large git repository or on a slow machine. To avoidthis problem, callgitstatus_query asynchronously as documented ingitstatus.plugin.zsh. This can be quite challenging.

The easiest way to take advantage of gitstatus from Bash is viagitstatus.prompt.sh. Install it as follows:

git clone --depth=1 https://github.com/romkatv/gitstatus.git~/gitstatusecho'source ~/gitstatus/gitstatus.prompt.sh'>>~/.bashrc

Users in China can use the official mirror on gitee.com for faster download.
中国大陆用户可以使用 gitee.com 上的官方镜像加速下载.

git clone --depth=1 https://gitee.com/romkatv/gitstatus.git~/gitstatusecho'source ~/gitstatus/gitstatus.prompt.sh'>>~/.bashrc

Alternatively, if you have Homebrew installed:

brew install romkatv/gitstatus/gitstatusecho"source$(brew --prefix)/opt/gitstatus/gitstatus.prompt.sh">>~/.bashrc

(If you choose this option, replace~/gitstatus with$(brew --prefix)/opt/gitstatus/gitstatusin all code snippets below.)

This will give you a basic yet functional prompt with git status in it. It'sover 10x faster than any alternative that can give you comparable prompt.

In order to customize your prompt, setPS1 at the end of~/.bashrc after sourcinggitstatus.prompt.sh. Insert${GITSTATUS_PROMPT} where you want git status to go. For example:

source~/gitstatus/gitstatus.prompt.shPS1='\w ${GITSTATUS_PROMPT}\n\$'# directory followed by git status and $/# (normal/root)

The expansion of${GITSTATUS_PROMPT} can contain the following bits:

If you'd like to change the format of git status, or want to have greater control over theprocess of assemblingPS1, you can copy and modify parts ofgitstatus.prompt.sh instead of sourcing the script. Your~/.bashrc mightlook something like this:

source~/gitstatus/gitstatus.plugin.shfunctionmy_set_prompt() {  PS1='\w'if gitstatus_query&& [["$VCS_STATUS_RESULT"== ok-sync ]];thenif [[-n"$VCS_STATUS_LOCAL_BRANCH" ]];then      PS1+="${VCS_STATUS_LOCAL_BRANCH//\\/\\\\}"# escape backslashelse      PS1+=" @${VCS_STATUS_COMMIT//\\/\\\\}"# escape backslashfi(( VCS_STATUS_HAS_STAGED"))&& PS1+='+'(( VCS_STATUS_HAS_UNSTAGED"))&& PS1+='!'(( VCS_STATUS_HAS_UNTRACKED"))&& PS1+='?'fi  PS1+='\n\$'shopt -u promptvars# disable expansion of '$(...)' and the like}gitstatus_stop&& gitstatus_startPROMPT_COMMAND=my_set_prompt

This snippet is sourcinggitstatus.plugin.sh rather thangitstatus.prompt.sh. The formerdefines low-level bindings that communicate with gitstatusd over pipes. The latter is a simplescript that uses these bindings to assemble git prompt.

Note: Bash bindings, unlike Zsh bindings, don't support asynchronous calls.

If there are no gitstatusd bindings for your shell, you'll need to get your hands dirty.Use the existing bindings for inspiration; rungitstatusd --help or read the same thing inoptions.cc.

gitstatusd reads requests from stdin and prints responses to stdout. Requests contain an ID anda directory. Responses contain the same ID and machine-readable git status for the directory.gitstatusd keeps some state in memory for the directories it has seen in order to serve futurerequests faster.

Zsh bindings andBash bindings start gitstatusd inthe background and communicate with it via pipes. Themes such asPowerlevel10k use these bindings to put git status inPROMPT.

Note that gitstatus cannot be used as a drop-in replacement forgit status command as it doesn'tproduce output in the same format. It does perform the same computation though.

The following benchmark results were obtained on Intel i9-7900X running Ubuntu 18.04 ina cleanchromium repository synced to9394e49a. Therepository was checked out to an ext4 filesystem on M.2 SSD.

Three functionally equivalent tools for computing git status were benchmarked:

• gitstatusd
• git withcore.untrackedcache enabled andcore.fsmonitor disabled
• lg2 -- a demo/example executable fromlibgit2 thatimplements a subset ofgit functionality on top of libgit2 API; for the purposes of thisbenchmark the subset is sufficient to generate the same data as the other tools

Every tool was benchmark in cold and hot conditions. Forgit the first run in a repository wasconsidered cold, with the following runs considered hot.lg2 was patched to compute results twicein a single invocation without freeing the repository in between; the second run was considered hot.The same patching was not done forgit becausegit cannot be easily modified to refresh inmemoryindex state between invocations; in fact, this limitation is one of the primary reasons developersuse libgit2.gitstatusd was benchmarked similarly tolg2 with two result computations in thesame invocation.

Two commands were benchmarked:status anddescribe.

In this benchmark all tools were computing the equivalent ofgit status. Lower numbers are better.

gitstatusd is substantially faster than the alternatives, especially on hot runs. Note that hot runsare of primary importance to the main use case of gitstatus in interactive shells.

The performance ofgit status fluctuated wildly in this benchmarks for reasons unknown to theauthor. Moreover, performance is sticky -- oncegit status settles around a number, it staysthere for a long time. Numbers as diverse as 295, 352, 663 and 730 had been observed on hot runs onthe same repository. The number in the table is the lowest (fastest or best) thatgit status hadshown.

In this benchmark all tools were computing the equivalent ofgit describe --tags --exact-matchto find tags that resolve to the same commit asHEAD. Lower numbers are better.

gitstatusd is once again faster than the alternatives, more so on hot runs.

Since gitstatusd doesn't have to print all staged/unstaged/untracked files but only reportwhether there are any, it can terminate repository scan early. It can also remember which fileswere dirty on the previous run and check them first on the next run to avoid the scan entirely ifthe files are still dirty. However, the benchmarks above were performed in a clean repository wherethese shortcuts do not trigger. All benchmarked tools had to do the same work -- check the statusof every file in the index to see if it has changed, check every directory for newly created files,etc. And yet, gitstatusd came ahead by a large margin. This section describes what it does thatmakes it so fast.

Most of the following comparisons are done against libgit2 rather than git because of the author'sfamiliarity with the former but not the with latter. libgit2 has clean, well-documented APIs and anelegant implementation, which makes it so much easier to work with and to analyze performancebottlenecks.

Under the benchmark conditions described above, the equivalent of libgit2'sgit_diff_index_to_workdir (the most expensive part ofstatus command) is 46.3 times faster ingitstatusd. The speedup comes from the following sources.

• gitstatusd uses more efficient data structures and algorithms and employs performance-consciouscoding style throughout the codebase. This reduces CPU time in userspace by 32x compared to libgit2.
• gitstatusd uses less expensive system calls and makes fewer of them. This reduces CPU time spentin kernel by 1.9x.
• gitstatusd can utilize multiple cores to scan index and workdir in parallel with almost perfectscaling. This reduces total run time by 12.4x while having virtually no effect on total CPU time.

The most resource-intensive part of thestatus command is finding the difference betweenindexandworkdir (git_diff_index_to_workdir in libgit2). Index is a list of all files in the gitrepository with their last modification times. This is an obvious simplification but it suffices forthis exposition. On disk, index is stored sorted by file path. Here's an example of git index:

This list needs to be compared to the list of files in the working directory. If any of the fileslisted in the index are missing from the workdir or have different last modification time, they are"unstaged" in gitstatusd parlance. If you rungit status, they'll be shown as "changes not stagedfor commit". Thus, any implementation ofstatus command has to callstat() or one of itsvariants on every file in the index.

In addition, all files in the working directory for which there is no entry in the index at all are"untracked".git status will show them as "untracked files". Finding untracked files requires someform of work directory traversal.

Let's see howgit_diff_index_to_workdir from libgit2 accomplishes these tasks. Here's its CPUprofile from 200 hot runs over chromium repository.

(The CPU profile was created withgperftools andrendered withpprof).

We can see__GI__lxstat taking a lot of time. This is thestat() call for every file in theindex. We can also identify__opendir,__readdir and__GI___close_nocancel -- glibc wrappersfor reading the contents of a directory. This is for finding untracked files. Out of the total 232seconds, 111 seconds -- or 47.7% -- was spent on these calls. The rest is computation -- comparingstrings, sorting arrays, etc.

Now let's take a look at the CPU profile of gitstatusd on the same task.

The first impression is that this profile looks pruned. This isn't an artifact. The profile wasgenerated with the same tools and the same flags as the profile of libgit2.

Since both profiles were generated from the same workload, absolute numbers can be compared. We cansee that gitstatusd took 62 seconds in total compared to libgit2's 232 seconds. System calls at thecore of the algorithm are clearly visible.__GI___fxstatat is a flavor ofstat(), and the otherthree calls --__libc_openat64,__libc_close and__GI___fxstat are responsible for openingdirectories and finding untracked files. Notice that there is almost nothing else in the profileapart from these calls. The rest of the code accounts for 3.77 seconds of CPU time -- 32 times lessthan in libgit2.

So, one reason gitstatusd is fast is that it has efficient diffing code -- very little time is spentoutside of kernel. However, if we look closely, we can notice that system calls in gitstatusd arealso faster than in libgit2. For example, libgit2 spent 72.07 seconds in__GI__lxstat whilegitstatusd spent only 48.82 seconds in__GI___fxstatat. There are two reasons for this difference.First, libgit2 makes morestat() calls than is strictly required. It's not necessary to statdirectories because index only has files. There are 25k directories in chromium repository (and 300kfiles) -- that's 25kstat() calls that could be avoided. The second reason is that libgit2 andgitstatusd use different flavors ofstat(). libgit2 useslstat(), which takes a path to the fileas input. Its performance is linear in the number of subdirectories in the path because it needs toperform a lookup for every one of them and to check permissions. gitstatusd usesfstatat(), whichtakes a file descriptor to the parent directory and a name of the file. Just a single lookup, lessCPU time.

Similarly tolstat() vsfstatat(), it's faster to open files and directories withopenat()from the parent directory file descriptor than with regularopen() that accepts full file path.gitstatusd takes advantage ofopenat() to open directories as fast as possible. It opens about 90%of the directories (this depends on the actual directory structure of the repository) from theimmediate parent -- the most efficient way -- and the remaining 10% it opens from the repository'sroot directory. The reason it's done this way is to keep the maximum number of simultaneously openfile descriptors bounded. libgit2 can have O(repository depth) simultaneously open file descriptors,which may be OK for a single-threaded application but can balloon to a large number when scans aredone by many threads simultaneously, like in gitstatusd.

There is no equivalent to__opendir or__readdir in the gitstatusd profile because it uses theequivalent ofuntracked cache fromgit. On the first scan of the workdir gitstatusd lists all files just like libgit2. But, unlikelibgit2, it remembers the last modification time of every directory along with the list ofuntracked files under it. On the next scan, gitstatusd can skip listing files in directories whoselast modification time hasn't changed.

To summarize, here's what gitstatusd was doing when the CPU profile was captured:

1. __libc_openat64: Open every directory for which there are files in the index.
2. __GI___fxstat: Check last modification time of the directory. Since it's the same as on thelast scan, this directory has the same list of untracked files as before, which is empty (therepository is clean).
3. __GI___fxstatat: Check last modification time for every file in the index that belongs to thisdirectory.
4. __libc_close: Close the file descriptor to the directory.

Here's how the very first scan of a repository looks like in gitstatusd:

(Some glibc functions are mislabel on this profile.explicit_bzero and__nss_passwd_lookup arein realitystrcmp andmemcmp.)

This is a superset of the previous -- hot -- profile, with an extrasyscall and string sorting fordirectory listing. gitstatusd usesgetdents64 Linux system call directly, bypassing the glibcwrapper that libgit2 uses. This is 23% faster. The details of this optimization can be found in aseparate document.

The diffing algorithm in gitstatusd was designed from the ground up with the intention of using itconcurrently from multiple threads. With a fast SSD,status is CPU bound, so taking advantage ofall available CPU cores is an obvious way to yield results faster.

gitstatusd exhibits almost perfect scaling from multithreading. Engaging all cores allows it toproduce results 12.4 times faster than in single-threaded execution. This is on Intel i9-7900X with10 cores (20 with hyperthreading) with single-core frequency of 4.3GHz and all-core frequency of4.0GHz.

Note:git status also uses all available cores in some parts of its algorithm whilelg2 doeseverything in a single thread.

Once the difference between the index and the workdir is found, we have a list ofcandidates --files that may be unstaged or untracked. To make the final judgement, these files need to be checkedagainst.gitignore rules and a few other things.

gitstatusd usespatched libgit2 for this step. This forkadds several optimizations that make libgit2 faster. The patched libgit2 performs more than twiceas fast in the benchmark as the original even without changes in the user code (that is, in thecode that uses the libgit2 APIs). The fork also adds several API extensions, most notable of whichis the support for multi-threaded scans. Iflg2 status is modified to take advantage of theseextensions, it outperforms the original libgit2 by a factor of 18. Lastly, the fork fixes a score ofbugs, most of which become apparent only when using libgit2 from multiple threads.

WARNING: Changes to libgit2 are extensive but the testing they underwent isn't. It isnot recommended to use the patched libgit2 in production.

• To compile: binutils, cmake, gcc, g++, git and GNU make.
• To run: Linux, macOS, FreeBSD, Android, WSL, Cygwin or MSYS2.

There are prebuiltgitstatusd binaries inreleases. When using the official shell bindingsprovided by gitstatus, the right binary for your architecture gets downloaded automatically.

If prebuilt binaries don't work for you, you'll need to get your hands dirty.

git clone --depth=1 https://github.com/romkatv/gitstatus.gitcd gitstatus./build -w -s -d docker

Users in China can use the official mirror on gitee.com for faster download.
中国大陆用户可以使用 gitee.com 上的官方镜像加速下载.

git clone --depth=1 https://gitee.com/romkatv/gitstatus.gitcd gitstatus./build -w -s -d docker

• If it says that-d docker is not supported on your OS, remove this flag.
• If it says that-s is not supported on your OS, remove this flag.
• If it tell you to install docker but you cannot or don't want to, remove-d docker.
• If it says that some command is missing, install it.

If everything goes well, the newly built binary will appear in./usrbin. It'll be picked upby shell bindings automatically.

When you update shell bindings, they may refuse to work with the binary you've built earlier. Inthis case you'll need to rebuild.

If you are using gitstatus throughPowerlevel10k, theinstructions are the same except that you don't need to clone gitstatus. Instead, change yourcurrent directory to/path/to/powerlevel10k/gitstatus (/path/to/powerlevel10k is the directorywhere you've installed Powerlevel10k) and run./build -w -s -d docker from there as describedabove.

It's currently neither easy nor recommended to package and distribute gitstatus. There are noinstructions you can follow that would allow you to easily update your package when new versions ofgitstatus are released. This may change in the future but not soon.

GNU General Public License v3.0. SeeLICENSE. Contributions are covered by the samelicense.