- Notifications
You must be signed in to change notification settings - Fork2.5k
Addgit_mempack_write_thin_pack
#6875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Thanks! Really useful addition; I'll review this shortly. 🙏 |
Mic92 commentedSep 20, 2024
@ethomson friendly ping. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for this. I made two very trivial minor suggestions in keeping with the library's styling. Otherwise, a great addition. 🙏
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Unlike existing functions, this produces a _thin_ packfile bymaking use of the fact that only new objects appear in themempack Object Database.A thin packfile only contains certain objects, but not its wholeclosure of references. This makes it suitable for efficientlywriting sets of new objects to a local repository, by avoidingmany small I/O operations.This relies on write_pack (e.g. git_packbuilder_write_buf) toimplement the "recency order" optimization step.Basic measurements comparing against the writing of individualobjects show a speedup during when writing large amounts ofcontent on machines with comparatively slow I/O operations,and little to no change on machines with fast I/O operations.
14e355d
tof9c35fb
CompareMic92 commentedSep 27, 2024
CI is green! |
9ddd72f
intolibgit2:mainUh oh!
There was an error while loading.Please reload this page.
Mic92 commentedSep 28, 2024
Question, will this patch be part of the v1.8.2 or later? |
I'm going to cut a release 🔜 |
Thank you@ethomson for reviewing and accepting! |
Ericson2314 commentedNov 11, 2024
Any ETA on this? |
Mic92 commentedNov 24, 2024
1.8.4 was released a while ago but this patch is no included. Are the patch releases made of a different branch? |
Ericson2314 commentedFeb 9, 2025
@Mic92 Ah I see it is in 1.9.0:https://github.com/libgit2/libgit2/blob/main/docs/changelog.md#v190 |
Unlike existing functions, this produces athin packfile by making use of the fact that only new objects appear in the mempack Object Database.
A thin packfile only contains certain objects, but not its whole closure of references. This makes it suitable for efficiently writing sets of new objects to a local repository, by avoiding many small I/O operations.
This relies on
write_pack
(e.g.git_packbuilder_write_buf
) to implement the "recency order" optimization step.Basic measurements comparing against the writing of individual objects show a speedup during when writing large amounts of content on machines with comparatively slow I/O operations, and little to no change on machines with fast I/O operations.
I have used this inNixOS/nix#11330, in conjunction with#6874