Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Future breaking changes in libgit2 v2.0#6988

ethomson started this conversation inAnnouncements
Dec 28, 2024· 2 comments· 8 replies
Discussion options

We expectlibgit2 v1.9 to be the last release in the libgit2 v1.x release line. libgit2 v2.0 will include a number of breaking changes.

  • SHA256 support
    SHA256 support will graduate from "experimental" to be a supported part of the library. This will include both API and ABI changes to support.

  • TLS v1.2 as a minimum
    libgit2 will remove support for HTTPS versions prior to TLS v1.2 and will update to the "intermediate" settingsdocumented by Mozilla.

  • libtool-compatible SONAME versioning
    Beginning with libgit2 v2.0, we will use SONAME versioning standards that align withlibtool versioning requirements. Meaning, we will update the SONAME major version number on incompatible ABI updates. In addition, we will attempt to keep more ABI compatibility between minor versions of the library.

  • Removing the chromium zlib built-in
    We will remove the ability for our cmake build system to compile a chromium zlib implementation in the "deps" directory. Users are welcome to build against chromium zlib, but should build it separately from libgit2, then link against it.

  • Removing the libssh2 embedded build system
    Similarly, we will remove the ability for our cmake build system to compile libssh2. Users are welcome to build against libssh2, but will need to build libssh2 separately from libgit2, then link against it.

  • WinHTTP deprecation
    WinHTTP will no longer be the default HTTP provider for Windows, and will be deprecated for behavioral compatibility across operating systems.

  • Object type cleanup
    Thegit_object_t enum contains items likeGIT_OBJECT_OFS_DELTA, which isnot an object type, it's a descriptor of an item in a packfile. Similarly, the APIgit_object_type_isloose is nonsensical. The former should not exist, and the latter will be updated to talk about type validity.

You must be logged in to vote

Replies: 2 comments 8 replies

Comment options

ethomson
Jan 3, 2025
Maintainer Author

Note that one thing that was mentioned in#6191 was the idea that carrying around largergit_oids that support SHA256 is wasteful for SHA1 repositories. This is, of course, true, but not something that we've addressed for v2.0.

I want to be a bit more concrete about this need and use cases before I write any code, but one can imagine something along the lines of:

typedefstructgit_sha1_oid {/** raw binary formatted id */unsignedcharid[GIT_OID_SHA1_SIZE];}git_sha1_oid;GIT_EXTERN(int)git_sha1_blob_lookup(git_blob**blob,git_repository*repo,constgit_sha1_oid*id);...intgit_sha1_blob_lookup(git_blob**out,git_repository*repo,constgit_sha1_oid*id){git_oidproxy;proxy.type=GIT_OID_SHA1;memcpy(&proxy,id->id,GIT_OID_SHA1_SIZE);returngit_blob_lookup(out,repo,&proxy);}

Similarly, for things that return or write to OIDs, we could do the opposite translation. This seems straightforward (and something we could probably even produce mechanically.) One could even imagine that you could do some dynamic loader trickery so that this "just works", but I'm not sure if that really makes sense. (It sounds complicated to me, and potentially brittle.)

There are a couple of issues here:

  1. It's not clear where this code should live. One could imagine that it lives within libgit2, a sidecar library that we provide, or even the caller's code. If there's broad need for this, then we should ship it; if not, it's probably not worth us investing in.
  2. More importantly,what do we do about callbacks? There are a handful of callbacks that give object IDs to callers. Presumably if you're using this odd API then you're all in ongit_sha1_oid or whatever we call it. I'd want to give yougit_sha1_oid in your callbacks as well. That's a little trickier; we could certainly build out proxies for all the things, but, meh.

Anyway. This is certainly possible, and I definitely want to give people the opportunity to save precious bytes. But without a clear need and a user who wants to help design this (or, better, help code it) then I'm going to hit pause ⏸️ here and say that we have an idea of what we would do, we're just waiting for the demand.

In the meantime, I intend to give people an opt-out to turnoff SHA256 support when they build to avoid those wasted bytes if they think that it's meaningful to them. (Basically inverting theEXPERIMENTAL_SHA256.)

You must be logged in to vote
0 replies
Comment options

One concern I have with SHA256 support is performance. In my code for constructing a commit history, I found thatgit_oid_equal() was a bottleneck: even when building withGIT_EXPERIMENTAL_SHA256 turned off, the switch statement ingit_oid_size was an extra step that could really add up in large repositories. I ended up just callingmemcmp directly, but this of course won't work once I actually need SHA256 support.

Is there a way we can have an approach where libgit2 knows that it's a SHA256 repo to begin with, and doesn't need to constantly check?

You must be logged in to vote
8 replies
@ethomson
Comment options

ethomsonJan 6, 2025
Maintainer Author

Interesting - I think that I misunderstood. You're saying that you saw a slowdown ingit_oid_equaleven when you're not compiling with Experimental SHA256 support? Thanks. This is helpful.

@Uncommon
Comment options

Yes, theswitch statement ingit_oid_size() ended up taking a bit of time even when it only had one case. It seems like maybe that should get optimized out, but it wasn't.

@ethomson
Comment options

ethomsonJan 6, 2025
Maintainer Author

Thanks. I've been digging in to the optimizations that clang is doing; I naively assumed that gcc and clang optimized roughly equivalently on arm64 and also did a reasonably poor job building a microbenchmark for this in the first place. I'll investigate and will ship a v1.9.1.

@ethomson
Comment options

ethomsonJan 7, 2025
Maintainer Author

It looks like you made this switch here —Uncommon/Xit@738ca60 — I'm curious if you happen to remember any more details? I ask because I'm having a bit of trouble replicating your report.

Digging a bit deeper - here:https://github.com/Uncommon/Xit/blame/e7ac31506a7666cedf2021dba00f5ed2a21e1d69/Xit/Repository/OID.swift you were using a custom wrapper overmemcpy, as the comment indicates, ourgit_oid_cmp did a byte-for-byte comparison, which was very surprising.

It looks like here:https://github.com/Uncommon/Xit/blame/9b05de72d0bcf9da4447e566ff88474c430ddc2d/Xit/Repository/OID.swift you started usinggit_oid_equal when you upgraded to libgit2 v1.5.0.

and here:Uncommon/Xit@738ca60 you moved back to amemcmp.

When I run a few milliongit_oid_equals against mainor v1.5.0, I don't see a speed difference. I also notice that libgit2 v1.5.0 already used thegit_oid_size. Also, removing thegit_oid_size and hardcoding it does not make a difference to me. So I wonder if something else is going on.

I'm not surprised thatmemcmp == 0 would be faster than a call into libgit2 forgit_oid_equal. So I'm not sure that I can get parity to amemcmp; gcc in particular is good at optimizingmemcmp. But I would like to speed it up, especially if there was a regression.

libgit2 v1.5.0 was 2.5 years ago. Any chance you were still using an x86 Mac then? I wonder if there are more obvious differences on x86 than arm. I can poke around at that hypothesis tomorrow.

@Uncommon
Comment options

I'm pretty sure I've had my M1 Mac for longer than that. What I recall is that I sawgit_oid_equal turn up in the profiling results, so I made the switch, and then it no longer appeared as a significant factor in the profiling. I don't recall what the numbers were like, though.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
None yet
2 participants
@ethomson@Uncommon

[8]ページ先頭

©2009-2025 Movatter.jp