Nerfstudio: A collaboration friendly studio for NeRFs(docs.nerf.studio) |
Nerfstudio: A collaboration friendly studio for NeRFs(docs.nerf.studio) |
For example, if you tell a NeRF that you have a camera at location (x, y, z) and pointing in direction (g, h, j), then the network will output the distance at which the ray emitted from the camera is expected to "hit" something and the RGB color of whatever is hit.
Doing things in this way enables rendering images at arbitrary resolutions (though rendering can be slow), and is naturally conducive to producing rotated views of objects or exploring 3D space. Also, at least theoretically, it should allow for more "compact" network architectures, as it does not need to output, say, a 512x512x3 image.
https://twitter.com/jeffdean/status/1524612819160735745?lang...
another paper toward Earth sized multi scale models (5 minutes)
And lo and behold I see something that looks like a 3d rendered view at the top of the entry for "Tower of London".
Except - it's just a video. A rendered video of a 360 view. Tried on Android and iPad. There's no freedom of movement, it plays a single look and even says "report this video" when you click the 3 dots.
Are the news reports wrong - or do the writers not know a video when they see one?
NeRF rendering latency is extremely low and frames are delivered over WebRTC. Which means that there's no fundamental problem to stream WebUI over the Internet.
Nerfstudio already supports all leading NeRF flavours and also includes their own. It instantly makes a staple for all future NeRF research.
Nerfstudio looks like something that is finally accessible enough that I will be able to start experimenting without having to spend enormous amounts of effort on tooling. Can't wait to check it out! It looks like some of the processing pipelines might be a good fit for merging with WebODM to make it even more friendly to work with.
My understanding is that you can't just take a few photographs, but you have to label them with their position and orientation, which is often not trivial.
But perhaps this difficulty has been solved already (?)
2. https://docs.nerf.studio/en/latest/quickstart/custom_dataset...
Also how does it compare to Nvidia Instant Nerf (performance etc)?