Gaussian Splats Tutorial

This tutorial shows the steps needed to generate a Gaussian splat and render it with V-Ray for Cinema 4D.

Generating a Gaussian Splat File

Gaussian Splatts take information from a series of panning images or videos of a scene or object and convert it into a 3-dimensional point cloud. The point cloud is then converted into Gaussians, which can hold information like size, color, and transparency. These Gaussians are then optimized until they are photorealistic, but they remain very light to render.

1. 1As a first step, you will need images or footage of a scene or object panning around at a moderate speed. For this tutorial, we have used the bicycle footage in the 3D Gaussian Splatting for Real-Time Radiance Field Rendering paper.You need to import the materials into software that generates Gaussian Splat files. There are numerous out there, but we have used postshot.Download the footage from the paper and import it (either as is or converted into separate images) in postshot.You can leave the settings at their default. They will produce a good result. Click Import to start the training.Click on the image to view it in full size.
2. 2
3. 3Once the training is done, go to the File menu and select Export Splat Model... This exports only the model, which is what we need. We will add cameras within V-Ray for Cinema 4D.Name the .ply file, select a destination, and click Save.Click on the image to view it in full size.

Rendering with V-Ray for Cinema 4D

1. 1Open Cinema 4D. In the Render Settings, set the Renderer to V-Ray.Go to the V-Ray menu > Geometry and select the V-Ray Gaussians.Click on the image to view it in full size.
2. 2Load the generated .ply file to the Gaussians geometry. Click on the icon and select the file we just exported.It imports with a very small Scale - 1cm. Increase it to around 100 cm.The axes of postshot are positioned differently from Cinema 4D. Thus, we need to adjust them to match Cinema 4D's environment. Set Flip Axis to Rotate XZ + Flip Y.
3. 3Next, Select the Plane and click on the V-Ray menu > Tools > Make Shadow Catcher. This tool automatically sets all the matte properties needed to transform the selected object into a Shadow catcher.Read more on the Make Shadow Catcher page.
4. 4Navigating the scene this way is not easy. Let's reduce the radius of the point cloud. In the Gaussians' settings set the Point Radius to 0.003. Leave the rest of the Preview settings as they are.
5. 5Next, Select the Plane and click on the V-Ray menu > Tools > Make Shadow Catcher. This tool automatically sets all the matte properties needed to transform the selected object into a Shadow catcher.Read more on the Make Shadow Catcher page.
6. 6Add a CameraLet's create a camera for our scene. If your viewport is positioned like in the previous step - looking at the bicycle, when you go to the V-Ray menu > Cameras > Physical Camera, the camera inherits the same position (with an applied V-Ray Physical Camera tag). This is a good way to easily set up your camera view.Here is our viewport preview and the rendered image.
7. 7Next, Select the Plane and click on the V-Ray menu > Tools > Make Shadow Catcher. This tool automatically sets all the matte properties needed to transform the selected object into a Shadow catcher.Read more on the Make Shadow Catcher page.
8. 8Add GeometryLet's now add two people talking in our scene. We will use the Chaos Cosmos Browser's library. Click on the image to view it in full size.
9. 9Add LightTo preview the geometry added to the Gaussian Splatts, you need to add a light.This scene is very cloudy; a soft atmospheric light would be ideal. Add a Dome light. This will serve as our final render light.Leave the Dome light settings at their default. Remember that the Gaussians geometry has light baked into it, but it does not emit any of that light. You always need to add a light source when adding additional geometry to a scene.
10. 10Add Shadow CatcherWe now have geometry in our scene, but it does not look as if it 'belongs' in the scene. Let's add a Shadowcatcher object under our two people talking to add realism. Create a Plane geometry and place it under the Cosmos assets. In this case, the needed coordinates are:
11. 11Next, Select the Plane and click on the V-Ray menu > Tools > Make Shadow Catcher. This tool automatically sets all the matte properties needed to transform the selected object into a Shadow catcher.Read more on the Make Shadow Catcher page.
12. 12Camera SetupThe current camera settings create a too-narrow preview; let's increase the field of view. Go to the Physical Camera tag. In the General tab, set the Focal Length (mm) to 31.If you need to adjust both the Focal Length (mm) and the Film Gate (mm) parameters, it is easier to use the Field of View property, as its value overrides both these parameters. However, we don't need further adjustments, and in this case, we don't need to use the Field of View property.
    We are only lighting the scene with a Dome light. Thus, to achieve proper lighting, we need to increase the camera's light sensitivity. Increase the Film Speed (ISO) value to 400.Click on the image to view it in full size
13. 13RenderAll we need to do now is render our animation. This will take a while, as every separate frame needs to be rendered. Consider using Chaos Cloud to speed up the process. Remember to set the Output Frame Range to Manual and set the range From 0 To 45. This way, when you begin a render, it will render out the entire animation.

Here is how the final animation looks. Remember, there is no actual geometry in this scene besides the two people talking. Using Gaussian Splatts, we can create entire scenes without much work. Try it yourself - create a scene for your next project using V-Ray Gaussians!

Reference

• Kerbl, B. et al. (2023) 3D Gaussian Splatting for Real-Time Radiance Field Rendering. tech. Available at: https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/