Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork940
Description
I'm working on a script using GitPython to loop over lots of Git repos, pull remote changes, etc...
At some point, I noticed that doingrepo.is_dirty() was opening some files which never got closed until the process exits, which in turns causes the tool to crash with "too many open files" after iterating over enough repos:
importosimportgit# Put here whatever Git repos you might have in the current folder# Listing more than one makes the problem more visiblerepos= ["ipset","libhtp"]raw_input("Check open files with `lsof -p %s | grep %s`"% (os.getpid(),os.getcwd()))fornameinrepos:repo=git.Repo(name)repo.is_dirty()delreporaw_input("Check open files again")# files are still open
I tried digging deeper down the GitPython stack, but couldn't find the actual cause.
In case that's helpful, below is the same as the previous snippet, but using directly the lowest-level gitdb objects I could find to open the files:
importosimportgitfromgitdb.utilimporthex_to_bin# Put here whatever Git repos you might have in the current folder# Listing more than one makes the problem more visiblerepos= ["ipset","libhtp"]raw_input("Check open files with `lsof -p %s | grep %s`"% (os.getpid(),os.getcwd()))raw_input("Check open files again")# files are still openfornameinrepos:repo=git.Repo(name)sha=hex_to_bin("71acab3ca115b9ec200e440188181f6878e26f08")fordbinrepo.odb._dbs:try:foritemindb._entities:# db._entities opens *.pack filesitem[2](sha)# opens *.idx filesexceptAttributeErrorase:# Not all DBs have this attributepassdelreporaw_input("Check open files again")# files are still open
But that's just where the files are open (well, in fact they are open even lower down but I didn't manage to go deeper), not where they are supposed to be closed.