Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

[FEAT] Add U-2-Net#1222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
BritishWerewolf wants to merge1 commit intohuggingface:main
base:main
Choose a base branch
Loading
fromBritishWerewolf:add-u2net

Conversation

BritishWerewolf
Copy link
Contributor

Prerequisites

These PRs are required because the processors lean heavily on the new features.

Add support for the U-2-Net architecture which is a mask generation model; useful for background removal.
Both U-2-Net and ISNet can be used with these processors.

Below is an example of what the model will achieve when using theBritishWerewolf/U-2-Netp model.

ModelOriginalMasked
U-2-NetpA photo of a Loungefly backpack with a Moana design on it. The background is a single shade of white.The same photo of the Loungefly backpack, however this time the white background has been removed and become transparent. The masking isn't perfect, and there is still a white halo around the image.
IS-NetA photo of a Loungefly backpack with a Moana design on it. The background is a single shade of white.The same photo of the Loungefly backpack, however this time the white background has been removed and become transparent. The masking isn't perfect, though better than U-2-Netp, and there is still a white halo around the image.

If you would like to run this code, you can do with theBritishWerewolf/U-2-Net collection of models.

// Create the processor.constprocessor=awaitAutoProcessor.from_pretrained(modelName);// Create the model.constmodel=awaitAutoModel.from_pretrained(modelName,{dtype:'fp32',});// Process the image.constimage=awaitRawImage.read('https://huggingface.co/ybelkada/segment-anything/resolve/main/assets/car.png');constprocessed_image=awaitprocessor(image);// Run the model.constoutput=awaitmodel({input:processed_image.pixel_values,});// Retrieve the mask, and scale it up to the size of the original image.constmask=awaitRawImage.fromTensor(output.mask).resize(image.width,image.height);// Apply the mask to the image and save it.constmaskedImage=awaitimage.putAlpha(mask);maskedImage.save('masked_image.png');

…gmentation.This supports both the U-2-Net and ISNet models.
@xenova
Copy link
Collaborator

Hi@BritishWerewolf 👋 Thanks for the PR!

I would imagine that the additional pre-processing needed introduces quite a large latency to model inference, especially because the necessary operations are implemented in JS as opposed to WASM/WebGPU. Is that correct?

Do you see any benefits of a model like this over a model listed in the recent background-removal pipeline update?#1216

@BritishWerewolf
Copy link
ContributorAuthor

Hey@xenova, I started work on this before the pipeline, but kept this PR as draft because I hoped to fix it down the line.
My current laptop just became too cumbersome to work with, because each iteration I was making just started to take a few minutes and it started adding up.

If you’re happy to leave this open as draft, then I will have a look into it when I buy a new laptop; but if you want to clean up the PRs and close this I completely understand (and this may be the better approach, I can reopen in futureif needed).

Thanks

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants
@BritishWerewolf@xenova
[8]ページ先頭
©2009-2025 Movatter.jp