AI can generate increasingly convincing 3D scenes. But creating a digital twin of a real environment is a different challenge. A model does not simply need to look realistic—it needs to reflect the structure, scale, and physical details of the original space.
Realsee’s founder, Xinchen Hui, recently explored what happens when GPT-6 Astra’s 3D capabilities are combined with real-world spatial data captured by the Realsee Galois 3D LiDAR scanner.
On one side was a rich spatial dataset: point clouds, high-resolution panoramas, depth maps, meshes, and pose information, all tied to real coordinates and real-world scale. On the other was GPT-6 Astra’s ability to interpret, reconstruct, and organize 3D environments.
The experiment points to a new possibility for digital twin technology: using AI to help turn captured spaces into more structured, editable, and simulation-ready digital twin assets.
Why Real-World Data Matters for Digital Twin Creation
Many AI 3D workflows begin with a small collection of photographs. Images can show what objects and surfaces look like, but they provide limited information about dimensions, depth, coordinates, and the relationships between different parts of a space.
When those details are missing, AI has to infer them. The result may look convincing while still containing incorrect proportions, misplaced objects, or inaccurate room connections.
A spatially accurate digital twin needs a stronger physical foundation.
A single Galois capture can produce multiple forms of spatial data, including panoramic images, HDR imagery, depth maps, cube faces, meshes, point clouds, and camera poses. Together, these outputs describe not only the appearance of a space but also its geometry, scale, position, and lighting context.
When GPT-6 Astra receives these inputs, it has less reason to guess. It can reconstruct the environment using references connected to the real site.
Testing AI Reconstruction in a Residential Space
The first part of the experiment used data from a real apartment.
With only basic inputs, GPT-6 Astra could generate a scene with a broadly correct structure. However, the materials relied on procedural approximations, some furniture proportions did not match the original space, and the details became less reliable under closer inspection.
After the point cloud, depth maps, and high-resolution panoramas were added, the reconstruction improved significantly. The AI could use the point cloud to review spatial geometry, depth data to check dimensions, and panoramas to reference actual materials, textures, and lighting.
Instead of building a visually plausible apartment from incomplete clues, the model could work from a more complete representation of the real environment.
This distinction is important for 3D digital twins. Without real spatial data, AI creates what it believes could exist. With measured spatial data, AI has a clearer basis for reconstructing what is actually there.
From an Industrial Scan to a Digital Twin in Manufacturing
The experiment then moved beyond residential space to a more demanding digital twin use case: an industrial facility.
Realsee’s captured VR asset already supported immersive viewing of the site. However, the original mesh represented the environment as a largely unified model. It did not automatically separate individual machines or assign equipment-level semantic information.
For an industrial or manufacturing digital twin, simulation, inspection, and robotic navigation require more than a visually complete mesh. Pipes, tanks, pumps, architectural surfaces, and major pieces of equipment often need to be identified and used as separate assets. Preparing them manually can require substantial modeling work.
Using Galois panoramas and point clouds as references, GPT-6 Astra demonstrated several useful capabilities.
Adding equipment information to the facility layout
Industrial floor plans often describe the building structure without labeling every piece of equipment. GPT-6 Astra used the captured spatial data to identify elements such as pumps, tanks, and pipelines, enriching the layout with equipment-level information.
Separating a unified mesh into editable assets
The model retained major architectural elements—such as walls, ceilings, floors, and fixed pipework—while separating large machines into individual objects. Repeated equipment could be handled as instances of the same asset, helping maintain consistency while making objects easier to edit, replace, or export.
Preparing assets for simulation and navigation
The separated models could also be prepared with collision geometry for robotics and navigation simulations. This creates a possible link between reality capture and digital twin simulation: the reconstructed facility becomes a structured environment in which robots can perceive obstacles, test routes, and rehearse tasks before operating on site.
For embodied AI and robot simulation, access to varied, reality-based environments is especially valuable. This workflow offers a possible route toward creating Sim-Ready assets for industrial and other non-residential spaces, where high-quality training and simulation data remain difficult to produce at scale.
Real Data Gives AI Less to Guess
AI models will continue to become more capable, but the reliability of a digital twin still depends heavily on the source data.
Galois does not provide GPT-6 Astra with a few disconnected images. It provides a coordinated spatial dataset with real geometry, real scale, and known viewpoints:
-
High-resolution panoramas provide references for materials, textures, and lighting.
-
Point clouds provide measurable geometry and spatial scale.
-
Depth maps help validate the dimensions and proportions of individual objects.
-
Pose information connects each capture position to a known coordinate and orientation.
These inputs give the reconstruction process evidence it can repeatedly compare against. The closer the source data is to the real environment, the more useful the resulting digital twin can become.
Could AI Change the Cost of Building Digital Twins?
Traditional digital twin production can involve scanning, manual modeling, asset separation, semantic labeling, validation, and delivery. For a complex industrial environment, that process may take weeks of specialized work.
The workflow explored in this experiment suggests a different direction:
Galois spatial capture → GPT-6 Astra-assisted reconstruction → structured, reusable 3D assets
If the approach continues to mature, AI-assisted digital twin creation could shift part of the workload from manual modeling to automated processing and human review. This may help teams create usable assets more quickly while preserving a direct connection to the captured site.
It could also expand the range of spaces for which digital twins are practical. Industrial facilities, museums, commercial buildings, and other complex environments may become easier to prepare for visualization, simulation, inspection, training, and spatial computing applications.
A New Direction for Reality-Based Digital Twins
This experiment does not represent a finished, one-click digital twin workflow. Object recognition, geometry repair, semantic accuracy, collision generation, and final validation still require further development and professional review.
But it demonstrates an important direction. AI is not only capable of generating new 3D worlds; when paired with reliable spatial capture, it may also help structure and reuse the real world as digital twin data.
The model provides the intelligence to interpret and transform the scene. Galois provides the physical foundation: real coordinates, real scale, and visual evidence captured from the site.
As AI capabilities become more widely available, that reality-based data may be what determines the quality and usefulness of the final digital twin.
The path is still at an early stage, but the possibility is clear: digital twin creation could begin moving from expensive, highly manual production toward a faster, data-driven workflow.
Interested in exploring reality-based digital twins with Realsee? Learn more about the Realsee Galois P4 or contact our team.
