Exploring emerging technologies for visual storytelling

Interactive scene with gaussian splat and Spline

Exploring emerging technologies for visual storytelling
Would it be possible to build an interactive environment so you can walk through the space or add extra objects?

Buildingblocks

# Hard- & Software / Service Cost Level 1-5
1 Mobile phone with decent camera €200 2
2 Luma Labs Free for now 2
2 Spline Free account 4
3 jQuery Free 4
4 Three.js Free 5
Step 01: Setting the scene
First, we set the scene. By simply stepping around the statue with the mobile phone, we film the surroundings.
Step 02: Create the Gaussian splat
As explained earlier, we can turn the recording into a Gaussian Splat. We can then import this into an html page and add text blocks if necessary.
Step 03: Import the splat into Spline

Without really elaborating on this and going into too much detail, thanks to the online 3-D tool Spline, it is now possible to import Gaussian Splat scenes into a 3D environment. From there, you can apply all the magic and power of Spline to the splat. You can add and modify objects, insert gravity, apply bounce boxes,... in short, anything you think is possible to obtain an interactive 3D environment. The quality of the overall splat reduces drastically, though.

Spline

Similar to traditional tools like Blender, 3DS Max, Autodesk Maya, and Google SketchUp; Spline also enables you to create 3d content, but there are some fundamental differences: it is web-based, it works in real-time and... most important, you can embed your 3d scenes/models on any website or web app.

https://spline.design/
Step 04: Implement on a webpage
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Spline test</title>
<script type="module" src="https://unpkg.com/@splinetool/viewer/build/spline-viewer.js"></script>
<style>
body, html {
height: 200%;
margin: 0;
overflow-y: scroll;
}
spline-viewer {
width: 100%;
height: 100vh;
display: block;
position: fixed;
}
</style>
</head>
<body>
<spline-viewer id="splineViewer" url="https://lab.idphotoagency.com/t/scene.splinecode"></spline-viewer>
<script>
document.addEventListener('DOMContentLoaded', () => {
const viewer = document.getElementById('splineViewer');
const initialPosition = { x: -0.38, y: 1.93, z: 0.13 };
const finalPosition = { x: -0.55, y: 0.08, z: 0.35 };
const maxScroll = document.documentElement.scrollHeight - window.innerHeight;

viewer.addEventListener('load', () => {
const camera = viewer.getCamera(); // Correct method to get the camera

window.addEventListener('scroll', () => {
const scrollTop = window.scrollY;
const scrollFraction = scrollTop / maxScroll;

const newX = initialPosition.x + (finalPosition.x - initialPosition.x) * scrollFraction;
const newY = initialPosition.y + (finalPosition.y - initialPosition.y) * scrollFraction;
const newZ = initialPosition.z + (finalPosition.z - initialPosition.z) * scrollFraction;

camera.position.set(newX, newY, newZ);
});
});
});
</script>
</body>
</html>
From here, everything is fairly straightforward. You can export the interactive Spline scene to a ‘splinecode’ file and implement it in your website using the provided api.
Beware, the loading time is longer and besides moving around the room with the mous, you can also use the key combination ‘Z-S-Q-D’ or ‘W-S-A-D’ when using qwerty to move through the room.

"The photographer is the Swiss Army knife of the visual content industry, versatile and indispensable, crafting moments into timeless stories."