Nerfstudio plug-and-play framework makes it easier to collaborate and develop interactive 3D scenes. (Video courtesy of Angjoo Kanazawa)Plug-and-play
Just a few years ago, Berkeley engineers showed us how they could easily turn images into a 3D navigable scene using a technology called Neural Radiance Fields, or NeRF. Now, another team has created a development framework to help speed up NeRF projects and make this technology more accessible to others.
Led by Angjoo Kanazawa, assistant professor of electrical engineering and computer sciences, the researchers have developed Nerfstudio, a Python framework that provides plug-and-play components for implementing NeRF-based methods, making it easier to collaborate and incorporate NeRF into projects.
“Advancements in NeRF have contributed to its growing popularity and use in applications such as computer vision, robotics, visual effects and gaming. But support for development has been lagging,” said Kanazawa. “The Nerfstudio framework is intended to simplify the development of custom NeRF methods, the processing of real-world data and interacting with reconstructions.”
This new framework is already helping engineers that employ interactive computer graphics in their work, specifically those creating 3D reconstructions in real-world settings. This includes roboticists who use NeRF for manipulation, motion planning, simulation and mapping, as well as gaming studios and news outlets that use interactive graphics to tell stories.
Since the introduction of NeRF, researchers worldwide have been working to improve the core technology. But this work is often performed by groups using proprietary repositories, making it difficult to share contributions with the larger NeRF community. Nerfstudio addresses this by providing a modular framework that “consolidates these research innovations.” In addition, it makes the associated code and data publicly available through open-source licensing.
Presently, 20 Berkeley engineers are actively contributing to Nerfstudio and helping to maintain it. And as many as 100 people outside the university have contributed to the core code since its launch in October 2022.
Learn more: Researchers create open-source platform for Neural Radiance Field development; Nerfstudio: A modular framework for Neural Radiance Field development (arXiv)