Turn 2D Footage into 3D scenes

Creating realistic 3D effects from 2D footage is a powerful technique for visual effects artists, enabling the addition of elements like mist, fog, depth of field, or even integrating 3D objects into a scene. This tutorial will guide you through the process of converting your 2D footage into a 3D environment using Fusion, allowing for greater creative flexibility in post-production.

Step 1: Setting Up the Tracker

1. Open your footage in Fusion.

2. Press Shift + Space, search for Tracker, and add it to your node tree.

3. If your version of Fusion defaults to the Intelli Track Tracker, replace it with a regular Point Tracker for more precise control.

4. Select high-contrast points in your footage for tracking.

5. Once selected, navigate to Tracking and use the middle button to track forward and backward.

Step 2: Stabilizing the Footage

1. Change the Operation from None to Match Move.

2. Set the Merge setting to Background Only.

3. To avoid unwanted movement at the edges, create a Merge node and add a Background node.

4. Adjust the Alpha of the background to create a transparent layer.

5. Increase the canvas size by modifying the Width and Height sliders to ensure no portions of the footage move out of frame.

Step 3: Creating the 3D Camera and Image Plane

1. Add a Camera 3D node and set the camera settings based on your footage specifications (e.g., 28mm lens for a Blackmagic Pocket Cinema 6K camera).

2. Create a Merge 3D node, a Render 3D node, and an Image Plane 3D.

3. Pipe the Image Plane into the Merge 3D node.

4. Enable Wireframe under the Image Plane settings and set X Rotation to -90°.

5. Adjust the Scale and Y Rotation to ensure alignment with the scene.

6. In Render 3D, change the Renderer Type to Software Renderer.

7. Adjust the Camera Transform settings to refine positioning.

Step 4: Projection Mapping for 3D Effects

1. Enable Camera Projection under the Camera 3D settings.

2. Change the Projection Mode from Ambient Light to Texture.

3. Add a Catcher Node to capture the image plane projection.

4. Integrate a Shape 3D node and set it to Sphere.

5. Increase the sphere’s size and position it close to the Image Plane 3D.

6. Adjust the Latitude Sliders to fine-tune the projection boundaries.

Step 5: Masking Foreground and Middle-Ground Elements

1. Add a Luma Keyer and pipe the footage into it.

2. Adjust the sliders to isolate the foreground elements and Invert the selection.

3. Use Garbage Matte to refine the mask further.

4. Add a Magic Mask, selecting the subject with a blue line and unwanted areas with a red line.

5. Track the mask forward and backward for accuracy.

6. Copy the Camera 3D node, rename it to Foreground Camera, and apply the mask.

7. Position the foreground elements in Merge 3D for proper depth alignment.

8. Repeat the process for middle-ground elements to build out the 3D depth structure.

Step 6: Enhancing Depth of Field

1. Go to Render 3D > Accumulation Effects and enable Depth of Field.

2. Adjust the Focal Plane and blur amount for realistic depth effects.

3. Refine by Decreasing Blur Intensity and increasing quality.

Step 7: Final Touches and Cleanup

1. Create a Clean Plate and set it to Ranges.

2. Pipe the Foreground Mask into the White Input of the Clean Plate node to remove unwanted objects.

3. Use Edge Feathering and Dilate/Erode nodes to refine edges.

4. Adjust Depth of Field blur to match the scene’s focus.

Now, you have successfully transformed a 2D video into a 3D scene, allowing for dynamic visual effects and enhanced creative control.

For high-quality textures and assets to elevate your projects, check out Archviz Supply.

Watch the full tutorial video below:

Video tutorial by Fusion Brew.