3D GAUSSIAN SPLATTING
This is a research project to gain insight on this new topic, understand the pipeline, and learn the applications.
Learnings
The main premise of 3D Gaussian Splatting is to recreate photorealistic depictions of physical scenes by encoding 2D images to create novel viewpoints. The pipeline first generates a point cloud using Structure-for-Motion for every image getting trained. For each point, a 3D gaussian is initialized at the center with the attributes opacity, covariance, mean, and an RGB color parameter. With each iteration of rendering, the gaussians are optimized to represent the scene geometry.
This new field of research is extremely interesting because being able to render something as high resolution and quickly as this helps researchers studying areas where humans can't reach.
DEMO
I performed a 3D Gaussian Splat of a classroom following this repo. This scene is trained on 84 frames and rendered with 7,000 point clouds. Some interesting aspects to note are the high resolution on things I didn't mean to capture in the video. For example, the clear writing on the chalkboard and the life-like detailing of the stuffed animal!