Yesterday,I saw a post in X asking for a self-hostable background remover service. I was thinking, can we make it work by using WebGPU? So it will run in the browser and doesn't require any server/queue to run After a couple of hours, I created this and published the source code on https://github.com/ducan-ne/remove-bg It's still new so welcome any ideas and contributions Powered by WebGPU and Transformer.js (RMBG V1.4 model) |
nothing interesting on the console.
https://www.reddit.com/r/StableDiffusion/comments/1dwkwrx/i_...
It's based on python and neural network modeling, and I was wondering if it's possible to run it via webgpu? Or run it based on wasm? No offense, just because it looks awesome.
In my understanding, It'll possible if the model's author build to https://onnxruntime.ai ONNX Runtime. And maybe the downside is user will need to download ton of data to their device, currently it's ~100-200mb
Will try to the deps simpler later, after all this is about a little time of work
Word of tip, you may not want to jump on every last suggestion from the peanut gallery instantly
Very cool, though!
Shameless plug: if you prefer API-based background removal we offer a super low cost, high quality option on https://pixian.ai
Despite all that, left-pad still gets > 1 million weekly downloads on npm.
> 1.1 License. > BRIA grants Customer a time-limited, non-exclusive, non-sublicensable, personal and non-transferable right and license to install, deploy and use the Foundation Model for the sole purpose of evaluating and examining the Foundation Model. > The functionality of the Foundation Model is limited. Accordingly, Customer are not permitted to utilize the Foundation Model for purposes other than the testing and evaluation thereof.
> 1.2.Restrictions. Customer may not: > 1.2.2. sell, rent, lease, sublicense, distribute or lend the Foundation Model to others, in whole or in part, or host the Foundation Model for access or use by others.
> The Foundation Model made available through Hugging Face is intended for internal evaluation purposes and/or demonstration to potential customers only.
My company runs a bunch of similar web-based services and plan to do a background remover at some stage, but as far as I know there's no current models with a sufficiently permissive license that can also feasibly download & run in browsers.
Everyone publishing AI model is actually acting as if they owned copyright over it and as such are sharing it with a license, but there's no legal basis for such claim at this point, it's all about pretending and hoping the law will be changed later on to make their claim valid.
Claim fair use
Release model
Claim copyright
Infinite copyright!
You might say that the models were legally trained since no law mandates consent for AI training. But no law says that models are copyrightable either.
>The TRIPS Agreement requires that copyright protection extends to databases and other compilations if they constitute intellectual creation by virtue of the selection or arrangement of their contents, even if some or all of the contents do not themselves constitute materials protected by copyright
But maybe that's just a me-problem.
Old habits die hard.
And the modern Internet implicitly assumes the end user is not on a metered connection. Websites are fucking massive these days.
You got me!
The model was 176 MB. Total pageload transferred 182 MB.
It doesn't seem like "Disable cache" in the DevTools empties the Cache Storage.
PS: WebGPU is the future
1. Background removal is working good on a lot of different types of images. This includes images with background, plain or white background, men, women, children, hair, and pets.
2. After background removal, the new image is warped in some areas. For example, I have a picture of a child eating ice-cream. The background was removed perfectly but left a lot of artifacts on the child. I can share those images for testing.
Please let me know if there are other areas I can test.
Feel free to raise an issue on Github to keep track progress, it helps open source a lot, or you can DM on x.com/duc__an anytime
nice work. I recently also published a WebGPU version of our browser-only background removal library. It's using the onnx-runtime under the hood. Weights are from isnet. It could also run birefnet – if there is some interest – however BirefNet weights are almost 1GB in size, which is a bit much to download I guess.
There is a blog post about it and also a CPU only version available.
https://img.ly/blog/browser-background-removal-using-onnx-ru...
Source is available at: https://github.com/imgly/background-removal-js
and on npm: @imgly/background-removal
Feel free to check it out!
If a browser's sandbox can't even protect against accidental resource exhaustion, I'd be very concerned about that as an intentional attack vector.
Please turn off the rolling animation for the duration timer. It looks really wrong when the numbers wind back (which they wouldn't do on a rotor) and when the trailing zeroes vanish.
I used to have all the powerful tools at my fingertips but now I feel like my Linux distro is slowly disappearing into a vortex of irrelevance.
Anyways, if you're looking for a cmd tool, rembg [0] is a pretty good one.
Why not fork the repo and repackage it as a CLI app if you really want it that way?
Original: https://imgur.com/a/NrEXfua
BG removed: https://imgur.com/a/JWKHVGE
Much of the background was untouched, and almost all of the actual data (the axis and bars) were removed instead.
Yeah I think it's because the quanlity of the model, hopefully we will have better quanlity in near future. I will see if anything I can do with the settings
I just spend my free time to make it much better based on your feedback
- Mobile support, tested on my iPhone
- GitHub README added
- Added medium zoom for easier viewing on Mobile
- The error banner makes it more friendly and easy troubleshooting
- Troubleshooting section in README
If you have any idea to make the UX better, please let me know, I'll appreciate it
Most probably it because of cache utilization and network overload, can simply be solved with clearing cache and creating a service worker for managing download of model and invalidating memory.
I've heard someone reported that Arc had a similar error
Firefox on Linux: Error: Unsupported device: "webgpu". Should be one of: wasm.
Chromium on Linux: Error: no available backend found. ERR: [webgpu] Error: Failed to get GPU adapter. You may need to enable flag "--enable-unsafe-webgpu" if you are using Chrome.
Passing the --enable-unsafe-webgpu flag results in the same error.
Can you try to go https://pmndrs.github.io/detect-gpu/ and pass the result here
It's a AMD Radeon RX 6600.
It worked on Chromium after passing --enable-unsafe-webgpu --enable-features=Vulkan
But it is great that there now is more offline tools. It would be great if there was a browser API that allowed the page to voluntarily go offline to guarantee that no data will be leaked.
Always happy to see other people exploring this niche
Good job!
Not that I'm blaming this tool for being worse than a $200+/yr product. If anything it's impressive how close it gets with so little code. And if you just want rough results on a large number of files it even looks superior
Other background removal tools would find me - but erase half the horse or erase his tack or his ears. This tool worked perfectly.
Very well done!
Memory can indeed be a problem, but at least if a tab becomes the largest single memory user on my system, the OOM killer will come for it first.
So if for CPU and memory browsers can lean on the OS for proper resource management but it’s not the case for GPU, maybe their WebGPU implementations aren’t ready for production yet.
This is a problem with popular Linux distributions, they do not have protection against application swapping out important system applications.
Linux has disk quotas and CPU scheduler but it doesn't have fair memory and swap management (or this is not configured out of the box). For example, desktop shell doesn't have protection against being swapped out.
A per-process real memory maximum that automatically invokes the OOM killer unless somehow opted out would probably be useful.
I think it's either running the model in the browser or a small part of it there. Maybe it's downloading parts of the model on the fly. But I kinda doubt it's all running on the server except for some simple RPC calls to the browser's WebGL.
Use client resources instead of server resources.
1. Load the page
2. Disconnect from the internet
3. Try to use the app without reconnecting
Edit: Good call! It's fully offline - I disabled the network in Chrome and it worked. Says it's 176MB. I think it must be downloading part of the model, all at once, but that's just a guess.
The 176MB is in storage which makes me think that my browser will hold onto it for a while. That's quite a lot. My browser really should provide a disk clearing tool that's more like OmniDiskSweeper than Clear History. If for instance it showed just the ones over 20MB, and my profile was using 1GB, at most it would be 50, a manageable amount to go through and clear the ones I don't need.
Google has started addressing this. I hope it becomes part of web standards soon.
https://developer.chrome.com/docs/ai/built-in
"Since these models aren't shared across websites, each site has to download them on page load. This is an impractical solution for developers and users"
The browser bundles might become quite large, but at least websites won't be.
