Introduction

Evercoast conducted a production to explore the viability of using RED KOMODO cameras for high-fidelity volumetric video capture.

The goal of this project was to validate that RED’s cinematic image quality, paired with Evercoast’s advanced Gaussian splatting reconstruction and playback pipeline, could produce photorealistic 4D human performance captures suitable for certain core markets that are important to both RED and Evercoast, including:

• Film & Television Production

• Visual Effects (VFX)

• Virtual and Augmented Reality Experiences (XR)

• Games and Interactive Content

While Evercoast’s platform has long supported professional multi-camera capture using its Mavericks software for camera control, recording, and live preview, this test was deliberately built using an ad hoc hardware setup. The intent was not to design a production-ready system, but rather to answer key questions about performance, workflow, and visual quality before deeper integration.

Purpose of the Test

This project was designed as a technical validation, not a final product deployment. The objectives included:

1. Image Quality Evaluation: Confirm that KOMODO cameras provide the fidelity and dynamic range desired in high-end volumetric rendering using Gaussian splats.

2. Multi-View Synchronization: Validate synchronization across 36 cameras to ensure perfect frame alignment during high-motion capture.

3. Workflow Analysis: Identify bottlenecks and opportunities to improve speed and efficiency before fully integrating RED cameras into Evercoast’s technology stack.

4. Optimizing Camera Count: While 36 cameras is substantial, it is far fewer than the 100+ cameras often seen in traditional volumetric rigs. Our goal was to strike a balance between production practicality and visual quality, testing the limits of reconstruction efficiency.

5. Future Integration Planning: Gather insights to guide the next phase, which will leverage RED Connect for IP-based streaming and Mavericks for seamless control, recording, and preview.

6. Benchmarking Against RealSense: Compare capture quality between a 29-camera RealSense D455 array and the 36-camera RED KOMODO array. While we knew the KOMODOs would produce superior imagery, this test was intended to quantify how much better the results were.

This project provided essential learnings to inform a next-generation production workflow that removes much of the complexity seen in this initial rig.

System Overview

The test rig consisted of 36 RED KOMODO cameras arranged to fully cover a 360-degree capture volume for complete human performance capture.

Capture Volume

• Diameter: 12 feet (3.66 meters)

• Vertical Poles: 8 evenly spaced, each supporting 4 cameras at staggered heights: 1 ft, 3 ft, 5 ft, and 7 ft

• Overhead Coverage: 4 additional cameras mounted at 10 ft for top-down coverage, eliminating blind spots.

This layout was designed to provide maximum coverage with significant overlap, a critical factor for robust 4D reconstruction.

Camera Configuration

Each KOMODO was outfitted with a straightforward kit:

1. Lens: Canon RF 15–35mm f/2.8 zoom

2. Settings: 4K/17:9, 30FPS, 1/240 shutter, ISO 800

3. Modules: RED KOMODO Expander Module and Link Adapter

4. Media: Recorded R3D LQ to RED PRO 512GB CFast card

5. Connectivity: USB-C to Ethernet dongle for network control

6. Monitoring & Sync:

• 3× 12G-SDI cables per camera: video out, genlock in, timecode in

• Brainstorm DXD-16 and AJA C10DA units for distributing genlock and timecode

• Decimator DMON-16S multiviewers feeding three monitors for live camera feeds

Networking & Control

A basic network was established using CAT-6A cabling and an unmanaged switch.

Each camera was connected via the RED KOMODO Link Adaptor to a central switch, which fed into a MacBook Pro running RED Control Pro. RED Control Pro enabled:

• Simultaneous start/stop of all cameras

• Basic remote camera adjustments

• Lens control for focus, iris, and zoom

This setup provided only baseline functionality. In future iterations, Evercoast’s Mavericks software will handle full array control, monitoring, and data management with a much cleaner IP-based network workflow through RED Connect.

Workflow

The workflow followed four key stages:

1. Capture: Cameras recorded synchronized footage using hardwired genlock and timecode to maintain exact frame alignment.

2. Data Offload: Footage was ingested from CFast cards and organized into a master dataset. Clips were trimmed to precise overlapping frame ranges with the CLI REDLine application and converted into Evercoast’s take format.

3. Cloudbreak Upload: Trimmed takes were uploaded into Cloudbreak, Evercoast’s spatial rendering platform. Cloudbreak streamlined distributed rendering and storage management.

4. Gaussian Splat Reconstruction: Evercoast’s Gaussian splatting implementation processed the data across multiple GPU-accelerated cloud instances hosted on AWS.

Comparing RED and RealSense

It may seem obvious to say, “of course a RED camera will outperform a RealSense for this kind of capture.” And yes — one is a $400 depth camera and the other is a $4,000 cinema camera. But cost alone doesn’t determine the right tool for a job. There are practical trade-offs around setup complexity, data volume, workflow efficiency, calibration, and rendering performance — and budget. A controlled technical comparison helps clarify not just that the RED performs better, but how and why, and whether the gains justify the additional hardware.

To provide a direct, side-by-side evaluation, both camera arrays were mounted and operated simultaneously within the same capture volume:

• RED KOMODO Array: 36 cameras in a full 360° configuration

• RealSense D455 Array: 29 cameras arranged for comprehensive coverage on the same pole positions

Each array recorded independently using its own workflow, with separate datasets managed, uploaded, and rendered through Cloudbreak. This allowed for a true apples-to-apples comparison without any overlap or blending of data.

Independent Reconstruction

The RED and RealSense datasets were processed exclusively and independently through Evercoast’s Gaussian splatting pipeline, ensuring that differences in output were driven purely by camera quality and capture fidelity, not workflow variations.

Quick Test — Big Insights

Although this rig was built quickly and using a more manual workflow, it did exactly what we wanted to do and that was to validate the viability of KOMODO cameras for professional volumetric capture.

Key Takeaways:

• The image quality was exceptional, delivering the dynamic range and resolution needed for premium content.

• RED cameras proved capable of precise multi-camera synchronization, a critical requirement for volumetric capture.

• The workflow revealed where future integrations with Mavericks and RED Connect will dramatically simplify setup and operation.

• The KOMODO captures retained subtle textures like skin pores, fabric weave, and hair strands, which were far less defined in the RealSense outputs.

  

• RED footage provided richer dynamic range and accurate color reproduction without banding or noise.

  

• The KOMODO array relied solely on monocular image data, eliminating the need for stereo depth pairs. Despite lacking dedicated depth sensors, the volumetric reconstructions were significantly cleaner and more stable.

• Motion tearing, jitter, and occlusion errors common in lower-cost cameras were virtually eliminated in the RED dataset.

  

We expected these results, but it’s always great to see expectations confirmed through real-world testing.

What’s Next

The next phase of our efforts will focus on creating a production-ready, turnkey, and easy-to-operate system, with:

• Centralized IP streaming via RED Connect

• Full array management through Mavericks

• Simplified cabling and hardware architecture

Sample Outputs

The result: photorealistic 4D sequences with true cinematic detail. The sample outputs from this test showcase:

• High-detail facial capture and natural expressions

• Smooth, artifact-free motion

• True-to-life textures and lighting

Conclusion

We absolutely validated that RED KOMODO cameras, paired with Evercoast’s Gaussian splatting pipeline, can achieve best-in-class volumetric capture for the entertainment industry.

While the rig was deliberately built as a test, the insights gained will guide a streamlined, user-friendly system.

With full integration into Mavericks and RED Connect, future deployments will feature seamless IP-based control, recording, and live preview — bringing cinematic volumetric capture to a new level of quality and efficiency.

Evercoast provides a comprehensive spatial data platform that brings real human performance into healthcare training, learning, and model development. The result is more human centered technology that strengthens confidence, communication, and clinical skill.

Healthcare organizations are already using spatial data in meaningful ways today. Below are three examples that demonstrate how real human movement, interaction, and expression are being used across mixed reality training, eLearning, and AI.

Mixed reality training that feels human

Johns Hopkins School of Nursing

Many nursing programs face a familiar challenge. Simulation labs are resource heavy, access is limited, and the emotional nuance of patient care can be difficult to recreate through lectures or role play. Johns Hopkins built the Extended Reality Learning Hub (XR Hub) as a new approach to clinical readiness on a Meta Quest headset. Evercoast provides the spatial data inside the XR Hub, allowing the team to bring real people into the experience with lifelike presence.

The core training module follows a patient, Mr. Jones, from admission to discharge. Students watch real conversations, observe bedside manners, and see the emotional side of care unfold. They notice details like a reassuring gesture, a moment of hesitation, or the tone of a difficult conversation. They also watch clinical procedures in three dimensional space, which helps students understand both the technical and interpersonal aspects of care. Because the experience runs in mixed reality, students anchor the spatial data in their own physical environment. This creates a strong sense of presence and bridges the gap between classroom instruction and real patient interaction.

For nursing students, this training is more memorable and more realistic than traditional methods. It gives them a fuller understanding of what patient care feels like, not only the checklist of tasks. The combination of mixed reality and real human performance strengthens confidence and prepares students for the emotional and relational moments of clinical care before they enter a hospital setting.

eLearning that enhances existing systems

Providence UPmersiv™

Not every healthcare organization is ready to implement mixed reality headsets at scale, and effective innovation does not need to wait for that moment. Providence took a different approach by using volumetric video to elevate the eLearning experience in their existing LMS platform. The goal was to improve soft skills and communication training and make it more engaging for clinicians at all levels.

To achieve this, Providence created an internal program called UPmersiv™, focused on developing modern training content and upskilling staff through immersive learning. Evercoast’s software is used to record realistic human scenarios that were then integrated into Providence’s eLearning courses built in Articulate. One module, “Using and Interpreting Body Language,” uses volumetric video clips to help clinicians observe subtle cues in patient and colleague communication. Learners watch how posture, facial expression, movement, and tone influence the effectiveness of care, and they reflect on how to apply these skills in their own practice.

This approach gave Providence a way to modernize training without replacing systems or requiring headsets. Volumetric content made the lessons feel more relatable and easier to retain, and it allowed training to scale across locations and schedules. The result was a more human and interactive learning experience inside a familiar delivery format, proving that volumetric capture can enhance learning wherever clinicians already are.

AI that learns from real movement

Hinge Health

As AI becomes more central to digital physical therapy and movement guidance, accuracy matters. Models trained on flat video often miss nuance—depth, biomechanics, body diversity, and edge cases.

Hinge Health uses Evercoast software to train and evaluate computer vision models that power their TrueMotion technology. By capturing human motion in 3D with precise spatial alignment, Evercoast provides a richer source of ground truth data.

Volumetric data helps models better understand joint angles, movement patterns, and variations across real bodies. It supports more accurate feedback during rehabilitation and better validation of model performance against real movement—not estimated approximations.

The result is more reliable AI for digital musculoskeletal care and a stronger foundation for future adaptive therapy systems.

The common thread

Across these programs, a clear pattern emerges: with one capture system, Evercoast records authentic human performance once and enables it to be used everywhere.

• Mixed Reality uses it to create lifelike presence in clinical scenarios.

• eLearning uses it to bring empathy and communication into traditional LMS platforms.

• AI uses it to train models on real movement rather than simulated data.

Volumetric / spatial data is not tied to one device or medium. It is a foundation for more human-centered technology in healthcare — making training more engaging, digital care more personal, and AI more accurate by grounding each experience in human reality.

As healthcare builds the next generation of learning, care, and intelligence, the most meaningful progress will come from technology that keeps humanity at the center. Evercoast enables that by making authentic human experience available wherever clinicians learn, practice, or innovate.

At Evercoast, we’ve been working to integrate our 4D Gaussian Splatting data into NVIDIA’s Omniverse ecosystem, with a focus on testing inside Isaac Sim. Static splats — single-frame Gaussian Splat representations — are already gaining traction in the Omniverse community for use in photorealistic scene capture and environment scanning. These are well-suited for static assets and environmental reconstruction.

Our focus, however, is on time-sequenced splats: dynamic, multi-frame volumetric video that captures human motion and real-world interactions with high fidelity. We’re exploring how well Omniverse and Isaac Sim can support this type of data for simulation, robotics, AI training, and digital twins.

Note: The observations and limitations described in this post are based on our internal testing with 4D Gaussian Splats in Omniverse and Isaac Sim. While consistent across our experiments, they may not reflect formal platform constraints or documented behavior.

Why 4D Gaussian Splats?

4D Gaussian Splats are a powerful way to represent real-world motion and appearance in volumetric form. Unlike traditional meshes or skeletal motion capture, splats preserve full surface and appearance detail, including soft tissue, fabric, hair, and complex lighting. They are time-sequenced, view-dependent, and inherently photorealistic.

While current implementations are not lightweight or performance-optimized, their value lies in fidelity. For simulation scenarios that demand realism — especially for perception-based AI and robotics training — splats offer a level of visual ground truth that other representations struggle to match.

Context: Omniverse and Isaac Sim

Omniverse is NVIDIA’s collaborative 3D platform for real-time simulation and design, built around the Universal Scene Description (USD) standard. Isaac Sim is a robotics simulation application built on Omniverse, providing tools for simulating sensors, environments, and robot behavior.

Our work has focused primarily within Isaac Sim, where accurate simulation of sensors and environments is critical. The question we’re exploring is whether 4D Gaussian Splats can become first-class citizens in that ecosystem — usable not just as static visual assets, but as integrated components in simulation, rendering, and sensor workflows depicting dynamic scenes and situations.

What We Found

1. Physics and Collision

Splats are not mesh-based and do not have explicit surface geometry. As expected, they cannot interact with physics engines in Isaac Sim out of the box. Collisions, dynamics, or physical presence would require supplemental geometry such as bounding boxes or proxy meshes.

Physics collisions for the default physics scene in Omniverse, with a splat and a mesh imported.

2. Sensor Integration (LIDAR and Visual SLAM)

Using the Carter robot in Isaac Sim, we tested whether splats are visible to simulation sensors. We’ve found that splats are not detected by LIDAR sensors, meaning they do not appear in depth scans or point clouds generated by non-visual sensors.

However, RGB cameras used in visual SLAM pipelines can see the splats, and we were able to extract sparse point clouds based on visual data. This suggests splats are usable in vision-based workflows but are currently invisible to depth or range-based systems.

Isaac Sim Carter Robot NVBlox Lidar example, showing the lack of depth computation for an imported splat.

3. Temporal Playback

We constructed a simple time-sequenced playback setup by manually keying the visibility of individual .usdz splat files. This worked for basic testing, but is not scalable. Current formats like .usdz do not natively support multi-frame sequences. A standardized approach for loading and playing splat sequences would greatly improve usability within both Omniverse and Isaac Sim.

Isaac Sim Carter Robot Visual SLAM example, showing the sparse point cloud representing the imported splat.

4. Rendering and Alpha Blending

When splats overlap or intersect with translucent materials, we observed rendering artifacts. These include hard edges and incorrect occlusion behavior. This likely stems from billboard-style rendering, where splats are rendered as 2D planes with depth-based sorting. Because they are not composited volumetrically, blending with other geometry — especially transparent objects — produces visual issues.

Manually keyed splat sequence for temporal playback

5. Shadows and Reflections

Splats do not currently cast or receive shadows and do not appear in reflections. Splats also cannot blend together when overlayed in the camera view, despite different world positions. This further reinforces the idea that they are treated as flat visual cards rather than volumetric content. This limitation restricts their use in photorealistic or physically accurate environments where lighting interaction is essential.

Interaction of a cube, casting shadows, over a mesh and a splat sequence. The mirrored floor shows no splat reflection, and two splats interacting shows billboard rasterization artifacts.

What This Could Enable in Omniverse and Isaac Sim

With deeper support, 4D Gaussian Splats could enable several high-value use cases within the NVIDIA simulation ecosystem:

• Sensor fidelity for AI perception: Splats could become visible to LIDAR, depth, and multimodal sensors, enabling synthetic data generation with realistic occlusions and surface interactions.

• Photorealistic digital twins: Time-sequenced splats could represent human activity inside complex digital twin environments with visual accuracy far beyond rigged meshes.

• Vision-language and scene understanding training: 4D splats provide a unique foundation for AI models to learn from temporally coherent, spatially rich representations of the real world.

• Simulation replay with real-world data: Captured 4D content could be dropped into simulation scenes for replay, annotation, or interaction with virtual agents or environments.

To unlock these possibilities, splats must be treated as more than static assets. They need full integration into Omniverse’s rendering, sensor, and simulation systems, with proper handling of time, transparency, and light interaction.

Where We Go from Here

This work is ongoing. We’re building tools, testing edge cases, and continuing to evaluate the role splats can play inside Isaac Sim. The questions we’re exploring include:

• Can splats be made visible to LIDAR and depth sensors?

• Will there be native support for time-sequenced playback in USD pipelines?

• Is there a path toward volumetric rendering with full shadow, reflection, and transparency support?

We’re excited about what’s possible and will continue to share what we learn. If you’re working on similar challenges in Omniverse, Isaac Sim, or volumetric content pipelines, we’d love to connect.

Get in touch!