~~we use gzip's "best compression"~~ (just saw that you also submitted the "best compression" PR in upstream)
we use our own bindata format, which is more efficient
no duplicate generated code, all bindata packages share the same "assetfs" code

reproducible build

79350b4

Copy link

Member

silverwind commentedJun 11, 2025•
edited
Loading

What's the benefit over using something more established likehttps://github.com/vearutop/statigz? I guess we would need a webpack gzip plugin, but might still be worth it.

Also, I wonder whether we could eliminate thego:generate step completely somehow.

Copy link

ContributorAuthor

wxiaoguang commentedJun 11, 2025

What's the benefit over using something more established likehttps://github.com/vearutop/statigz? I guess we would need a webpack gzip plugin, but might still be worth it.

It will bloat the binary about 20MB+

Also, I wonder whether we could eliminate thego:generate step completely somehow.

Not this PR's scope.

Copy link

Member

silverwind commentedJun 11, 2025•
edited
Loading

It will bloat the binary about 20MB+

If we make webpack write the.gz files and delete the original output files, I guess output size should be pretty similar.

Copy link

ContributorAuthor

wxiaoguang commentedJun 11, 2025

It will bloat the binary about 20MB+
If we make webpack write the.gz files and delete the original assets, I guess output size should be pretty similar.

There is no feasible plan at the moment, see the discussions in#26533

Copy link

ContributorAuthor

wxiaoguang commentedJun 11, 2025

Any other questions?

lunny reviewed

Jun 11, 2025

View reviewed changes

modules/assetfs/embed.goShow resolvedHide resolved

lunny added this to the1.25.0 milestone

Jun 11, 2025

This comment was marked as outdated.

lunny approved these changes

Jun 11, 2025

View reviewed changes

GiteaBot added lgtm/need 1This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2This PR needs two approvals by maintainers to be considered for merging. labels

Jun 11, 2025

fine tune

216e122

Copy link

ContributorAuthor

wxiaoguang commentedJun 12, 2025

Hmm, found another problem with**, will fix in a new commit.

fix embed

9981619

github-actionsbot added the modifies/cliPR changes something on the CLI, i.e. gitea doctor or gitea admin label

Jun 12, 2025

Copy link

ContributorAuthor

wxiaoguang commentedJun 12, 2025

Hmm, found another problem with**, will fix in a new commit.

Made more fixes in9981619

fix lint

213cbcd

wxiaoguang requested a review fromhiifong

June 12, 2025 02:10

hiifong approved these changes

Jun 12, 2025

View reviewed changes

GiteaBot added lgtm/doneThis PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1This PR needs approval from one additional maintainer to be merged. labels

Jun 12, 2025

wxiaoguang added2 commits

June 12, 2025 11:00

use go 1.24.4

5ef3413

Merge branch 'main' into refactor-embed-gzip

5222a16

wxiaoguangenabled auto-merge (squash)

June 12, 2025 03:05

Merge branch 'main' into refactor-embed-gzip

7ae6aa5

wxiaoguangdisabled auto-merge

June 12, 2025 03:26

wxiaoguang added the reviewed/wait-mergeThis pull request is part of the merge queue. It will be merged soon. label

Jun 12, 2025

Merge branch 'main' into refactor-embed-gzip

199905c

wxiaoguangenabled auto-merge (squash)

June 12, 2025 03:35

wxiaoguang merged commit65986f4 intogo-gitea:main

Jun 12, 2025

26 checks passed

GiteaBot removed the reviewed/wait-mergeThis pull request is part of the merge queue. It will be merged soon. label

Jun 12, 2025

lunny mentioned this pull request

Jun 12, 2025

Use go:embed for embedding files instead of vfsgen#17352

Closed

wxiaoguang deleted the refactor-embed-gzip branch

June 12, 2025 06:44

zjjhot added a commit to zjjhot/gitea that referenced this pull request

Jun 13, 2025

Merge remote-tracking branch 'giteaofficial/main'

1327a6a

* giteaofficial/main:  [skip ci] Updated translations via Crowdin  Improve the performance when detecting the file editable (go-gitea#34653)  Fix various problems (go-gitea#34708)  Refactor embedded assets and drop unnecessary dependencies (go-gitea#34692)  Bump minimum go version to 1.24.4 (go-gitea#34699)  Update JS deps (go-gitea#34701)  Fix markdown wrap (go-gitea#34697)  [skip ci] Updated translations via Crowdin  frontport changelog (go-gitea#34689)  Improve instance wide ssh commit signing (go-gitea#34341)

Copy link

Member

silverwind commentedJun 13, 2025

We could consider replacing gzip with zstd for more size savings and better page loading performance. 71% of browsers globally supportcontent-encoding: zstd.

zstd encoding currently needs a third-party module, but it's very likely that it will be added in golang stdlib at some point:golang/go#62513.

Copy link

ContributorAuthor

wxiaoguang commentedJun 13, 2025•
edited
Loading

We could consider replacing gzip with zstd for more size savings and better page loading performance.

Then the key point is not here, but the "gzip middleware"

71% of browsers globally supportcontent-encoding: zstd.

IMO it is still new, the supported browsers were released in recent year

zstd encoding currently needs a third-party module, but it's very likely that it will be added in golang stdlib at some point:golang/go#62513.

TBH I don't have any expectations for Golang's feature, when it is ready, we use it; if it isn't ready, we use others.

Copy link

Member

silverwind commentedJun 13, 2025•
edited
Loading

Then the key point is not here, but the "gzip middleware"

In any case, now that we have all this compression code in first-party, it should be easy to replace the encoding.

IMO it is still new, the supported browsers were released in recent year

I think the value will be > 80% by the time 1.25 releases.

TBH I don't have any expectations for Golang's feature, when it is ready, we use it; if it isn't ready, we use others.

Me neither, I expect the go feature earliest in a year or two. But it's still good to know that we can eventually eliminate the third-party dependency.

Copy link

Member

silverwind commentedJun 13, 2025

I think the value will be > 80% by the time 1.25 releases.

Maybe not, given that Safari lacks support:https://caniuse.com/zstd. But I guess the usage share of Safari users of gitea may be lower then the global average 😆.

wxiaoguang mentioned this pull request

Jun 15, 2025

Clean bindata#34728

Merged

wxiaoguang added a commit that referenced this pull request

Jun 16, 2025

Clean bindata (#34728)

24ce205

Follow#34692

DennisRasey pushed a commit to DennisRasey/forgejo that referenced this pull request

Jun 17, 2025

[gitea] week 2025-22 cherry pick (gitea/main -> forgejo) (#8198)

15bb6b7

## Checklist- [x] go to the last cherry-pick PR (forgejo/forgejo#8040) to figure out how far it went: [gitea@d5bbaee64e](go-gitea/gitea@d5bbaee)- [x] cherry-pick and open PR (forgejo/forgejo#8198)- [ ] have the PR pass the CI- end-to-end (specially important if there are actions related changes)  - [ ] add `run-end-to-end` label  - [ ] check the result- [ ] write release notes- [ ] assign reviewers- [ ] 48h later, last call- merge 1 hour after the last call## Legend- ❓ - No decision about the commit has been made.- 🍒 - The commit has been cherry picked.- ⏩ - The commit has been skipped.- 💡 - The commit has been skipped, but should be ported to Forgejo.- ✍️ - The commit has been skipped, and a port to Forgejo already exists.## Commits- 🍒 [`gitea`](go-gitea/gitea@17cfae8) -> [`forgejo`](https://codeberg.org/forgejo/forgejo/commit/6397da88d30de0a470dabadb8e27fbb202d75458) Hide href attribute of a tag if there is no target_url ([gitea#34556](go-gitea/gitea#34556))- 🍒 [`gitea`](go-gitea/gitea@b408bf2) -> [`forgejo`](https://codeberg.org/forgejo/forgejo/commit/46bc899d57515fc5349e9113e92da2e4b0d93c75) Fix: skip paths check on tag push events in workflows ([gitea#34602](go-gitea/gitea#34602))- 🍒 [`gitea`](go-gitea/gitea@9165ea8) -> [`forgejo`](https://codeberg.org/forgejo/forgejo/commit/04332f31bfd8a1c0e8676e4764d44e087f1ccc30) Only activity tab needs heatmap data loading ([gitea#34652](go-gitea/gitea#34652))- 🍒 [`gitea`](go-gitea/gitea@3f7dbbd) -> [`forgejo`](https://codeberg.org/forgejo/forgejo/commit/2a9019fd0491684cdeab6d50a16e5cffaef5508b) Small fix in Pull Requests page ([gitea#34612](go-gitea/gitea#34612))- 🍒 [`gitea`](go-gitea/gitea@497b83b) -> [`forgejo`](https://codeberg.org/forgejo/forgejo/commit/9a83cc7bad79fe79447bf6e3cb3144292f922ebb) Fix migration pull request title too long ([gitea#34577](go-gitea/gitea#34577))## TODO- 💡 [`gitea`](go-gitea/gitea@6b8b580) Refactor container and UI ([gitea#34736](go-gitea/gitea#34736))  Packages: Fix for container, needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@bbee652) Prevent duplicate form submissions when creating forks ([gitea#34714](go-gitea/gitea#34714))  Fork: Fix, needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@d21ce9f) Improve the performance when detecting the file editable ([gitea#34653](go-gitea/gitea#34653))  LFS: Performance improvement - needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@8fed27b) Fix various problems ([gitea#34708](go-gitea/gitea#34708))  Various: Fixes, tests missing.------- 💡 [`gitea`](go-gitea/gitea@c9505a2) Improve instance wide ssh commit signing ([gitea#34341](go-gitea/gitea#34341))  CodeSign: Nice feature - needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@fbc3796) Fix pull requests API convert panic when head repository is deleted. ([gitea#34685](go-gitea/gitea#34685))  Pull: Fix, needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@1610a63) Fix commit message rendering and some UI problems ([gitea#34680](go-gitea/gitea#34680))  Various Fixes - needs carefull merge.------- 💡 [`gitea`](go-gitea/gitea@0082cb5) Fix last admin check when syncing users ([gitea#34649](go-gitea/gitea#34649))  oidc: fix "first user is always admin". Needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@c6b2cbd) Fix footnote jump behavior on the issue page. ([gitea#34621](go-gitea/gitea#34621))  Issues: Fix Markdown rendering. Needs carefull merge------- 💡 [`gitea`](go-gitea/gitea@7a59f5a) Ignore "Close" error when uploading container blob ([gitea#34620](go-gitea/gitea#34620))  No issue, no test.------- 💡 [`gitea`](go-gitea/gitea@6d0b240) Keeping consistent between UI and API about combined commit status state and fix some bugs ([gitea#34562](go-gitea/gitea#34562))  Next PR in Commit-Status story.------- 💡 [`gitea`](go-gitea/gitea@f604144) Refactor FindOrgOptions to use enum instead of bool, fix membership visibility ([gitea#34629](go-gitea/gitea#34629))  Just for a common sense here: How should I consider refactorings?------- 💡 [`gitea`](go-gitea/gitea@cc942e2) Fix GetUsersByEmails ([gitea#34643](go-gitea/gitea#34643))  User: Seems to fix email validation - but seems not to be finished.------- 💡 [`gitea`](go-gitea/gitea@7fa5a88) Add `--color-logo` for text that should match logo color ([gitea#34639](go-gitea/gitea#34639))  UI: Nice idea - can we adapt this?------- 💡 [`gitea`](go-gitea/gitea@47d69b7) Validate hex colors when creating/editing labels ([gitea#34623](go-gitea/gitea#34623))  Label: Color validation but needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@108db0b) Fix possible pull request broken when leave the page immediately after clicking the update button ([gitea#34509](go-gitea/gitea#34509))  Nice fix for a bug hard to trace down.  Needs careful merge & think about whether a test is possible.------- 💡 [`gitea`](go-gitea/gitea@79cc369) Fix issue label delete incorrect labels webhook payload ([gitea#34575](go-gitea/gitea#34575))  Small fix but would expect a test, showing what was fixed.------- 💡 [`gitea`](go-gitea/gitea@fe57ee3) fixed incorrect page navigation with up and down arrow on last item of dashboard repos ([gitea#34570](go-gitea/gitea#34570))  Small & simple - but tests are missing.------- 💡 [`gitea`](go-gitea/gitea@4e47148) Remove unnecessary duplicate code ([gitea#34552](go-gitea/gitea#34552))  Fix arround "Split GetLatestCommitStatus".------- 💡 [`gitea`](go-gitea/gitea@c5e78fc) Do not mutate incoming options to SearchRepositoryByName ([gitea#34553](go-gitea/gitea#34553))  Large refactoring to simplify options handling. But needs careful merge.------- 💡 [`gitea`](go-gitea/gitea@f48c013) Fix/improve avatar sync from LDAP ([gitea#34573](go-gitea/gitea#34573))  Nice fix but needs test.------- 💡 [`gitea`](go-gitea/gitea@e8d8984) Fix some trivial problems ([gitea#34579](go-gitea/gitea#34579))  Various fixes, tests missing.------## Skipped- ⏩ [`gitea`](go-gitea/gitea@637070e) Fix container range bug ([gitea#34725](go-gitea/gitea#34725))------- ⏩ [`gitea`](go-gitea/gitea@0d3e995) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@28debdb) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@dcc9206) Raise minimum Node.js version to 20, test on 24 ([gitea#34713](go-gitea/gitea#34713))------- ⏩ [`gitea`](go-gitea/gitea@bc28654) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@65986f4) Refactor embedded assets and drop unnecessary dependencies ([gitea#34692](go-gitea/gitea#34692))------- ⏩ [`gitea`](go-gitea/gitea@18bafcc) Bump minimum go version to 1.24.4 ([gitea#34699](go-gitea/gitea#34699))------- ⏩ [`gitea`](go-gitea/gitea@8d135ef) Update JS deps ([gitea#34701](go-gitea/gitea#34701))------- ⏩ [`gitea`](go-gitea/gitea@d5893ee) Fix markdown wrap ([gitea#34697](go-gitea/gitea#34697))  - gitea UI specific specific------- ⏩ [`gitea`](go-gitea/gitea@06ccb3a) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@94db956) frontport changelog ([gitea#34689](go-gitea/gitea#34689))------- ⏩ [`gitea`](go-gitea/gitea@d5afdcc) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@e9f5105) Migrate to urfave v3 ([gitea#34510](go-gitea/gitea#34510))  already in Forgejo - seehttps://codeberg.org/forgejo/forgejo/pulls/8035------- ⏩ [`gitea`](go-gitea/gitea@2c341b6) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@92e7e98) Update x/crypto package and make builtin SSH use default parameters ([gitea#34667](go-gitea/gitea#34667))------- ⏩ [`gitea`](go-gitea/gitea@7b39c82) Fix "oras" OCI client compatibility ([gitea#34666](go-gitea/gitea#34666))  Already in forgejo - seehttps://codeberg.org/forgejo/forgejo/issues/8070------- ⏩ [`gitea`](go-gitea/gitea@1fe652c) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@a9a705f) Fix missed merge commit sha and time when migrating from codecommit ([gitea#34645](go-gitea/gitea#34645))  Migration: Seems to be an important fix, but no tests.  As I know@earl-warren worked hard on migration, is this still relevant to us?------- ⏩ [`gitea`](go-gitea/gitea@1e0758a) [skip ci] Updated translations via Crowdin------- ⏩ [`gitea`](go-gitea/gitea@f6f6aed) Update JS deps, regenerate SVGs ([gitea#34640](go-gitea/gitea#34640))------- ⏩ [`gitea`](go-gitea/gitea@aa2b3b2) Misc CSS fixes ([gitea#34638](go-gitea/gitea#34638))  - gitea UI specific specific------- ⏩ [`gitea`](go-gitea/gitea@b38f2d3) add codecommit to supported services in api docs ([gitea#34626](go-gitea/gitea#34626))------- ⏩ [`gitea`](go-gitea/gitea@74a0178) add openssh-keygen to rootless image ([gitea#34625](go-gitea/gitea#34625))  already in Forgejo - seehttps://codeberg.org/forgejo/forgejo/issues/6896------- ⏩ [`gitea`](go-gitea/gitea@5b22af4) bump to alpine 3.22 ([gitea#34613](go-gitea/gitea#34613))------- ⏩ [`gitea`](go-gitea/gitea@9e0e107) Fix notification count positioning for variable-width elements ([gitea#34597](go-gitea/gitea#34597))  - gitea UI specific specific------- ⏩ [`gitea`](go-gitea/gitea@e5781ce) Fix margin issue in markup paragraph rendering ([gitea#34599](go-gitea/gitea#34599))  - gitea UI specific specific------- ⏩ [`gitea`](go-gitea/gitea@375dab1) Make pull request and issue history more compact ([gitea#34588](go-gitea/gitea#34588))  - gitea UI specific specific------- ⏩ [`gitea`](go-gitea/gitea@2a1585b) Refactor some tests ([gitea#34580](go-gitea/gitea#34580))------<details><summary><h2>Stats</h2></summary><br>Between [`gitea@d5bbaee64e`](go-gitea/gitea@d5bbaee) and [`gitea@6b8b580218`](go-gitea/gitea@6b8b580), **55** commits have been reviewed. We picked **5**, skipped **28** (of which **3** were already in Forgejo!), and decided to port **22**.</details>Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>Co-authored-by: NorthRealm <155140859+NorthRealm@users.noreply.github.com>Co-authored-by: TheFox0x7 <thefox0x7@gmail.com>Co-authored-by: endo0911engineer <161911062+endo0911engineer@users.noreply.github.com>Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>Reviewed-on:https://codeberg.org/forgejo/forgejo/pulls/8198Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>Co-authored-by: Michael Jerger <michael.jerger@meissa-gmbh.de>Co-committed-by: Michael Jerger <michael.jerger@meissa-gmbh.de>