At the 19th European Conference on Computer Vision (ECCV 2026) in Malmö, Sweden, Realsee presented Argus, a deep learning model that reconstructs metric-scale indoor 3D scenes from an unordered set of sparse panoramic photos in a single feed-forward pass. Alongside the architecture, we introduced Realsee3D, a benchmark dataset encompassing 10,000 indoor properties, to bridge real-world capture conditions with dense geometric learning.
Reconstructing an indoor scene from a few panoramic images requires recovering accurate camera positions and physical surfaces in a common coordinate system. This fundamental challenge is the focus of our research, Argus: Metric Panoramic 3D Reconstruction for Indoor Scenes, conducted by researchers from Realsee in collaboration with Quanzhou University of Information Engineering.
Given a sparse, unordered set of indoor panoramas, Argus predicts camera poses and metric depth to generate a globally consistent point cloud without iterative optimization. The architecture couples learned reference-view selection with panoramic geometric factorization. Detailed resources, code, and demos are available on the Argus Project Page and the official Springer Nature chapter (pre-print also on arXiv).
Why Panoramic Metric Reconstruction Matters
Panoramas provide an efficient mechanism for indoor spatial capture. A modest number of stationary viewpoints can span an entire residential layout, with each 360-degree exposure capturing a complete horizontal field of view. This format is widely adopted in virtual real estate walkthroughs and commercial digital twins.
However, reconstructing sparse panoramas into physical 3D representations poses unique hurdles:
- Unordered capture: Unlike video streams, independent panoramas lack continuous motion trajectories.
- Narrow visual overlap: Adjacent rooms frequently share visibility only across narrow doorways.
- Physical scale requirements: Downstream spatial computing, robotics, and architectural workflows require real-world dimensions (meters) rather than arbitrary relative scale.
Argus addresses scale ambiguity natively through direct metric depth and translation supervision, eliminating the need for post-hoc scale alignment to external ground truth.
The Realsee3D Dataset: 10,000 Real and Synthetic Scenes
To support metric multi-view training, we developed Realsee3D, combining professional LiDAR scans of furnished residences with photorealistic synthetic properties.
| Subset | Indoor Scenes | Rooms | Unique Viewpoints |
|---|---|---|---|
| Real (LiDAR capture) | 1,000 | 9,483 | 24,263 |
| Synthetic (UE5 ray-tracing) | 9,000 | 86,479 | 274,810 |
| Total | 10,000 | 95,962 | 299,073 |
The real subset was acquired using Realsee's proprietary Galois 3D LiDAR scanning hardware, yielding RAW-domain equirectangular projection (ERP) HDR panoramas, sparse metric LiDAR depth, and globally registered poses. The synthetic subset models real-world floor-plan layouts across 100,000+ assets with Unreal Engine 5 ray-tracing, introducing diverse lighting conditions without sensor artifacts. Researchers can request dataset access via the Realsee3D Portal.
Core Method Innovations
1. Learned Reference-View Selection
In reference-based multi-view systems, one camera must define the coordinate anchor. Selecting an arbitrary first image can anchor the entire coordinate frame to an isolated peripheral viewpoint, compounding pose drift. Argus incorporates a lightweight Covisibility Transformer that scores all input views based on visual context, identifying a topologically central anchor view to achieve approximate permutation equivariance.
2. Panoramic Geometric Factorization
For an equirectangular image, every pixel maps deterministically to a ray on the unit sphere. Argus explicitly factorizes the 2D-to-3D projection pipeline across camera-space points, rotated points, and shared world coordinates. By applying joint forward and inverse geometric losses across depth, pose, and point heads during training, supervision propagates symmetrically across representations.
Experimental Results on the Realsee3D Benchmark
We evaluated Argus against panoramic adaptations of leading architectures, including MapAnything360, VGGT360, and π³360 under identical data splits and training schedules.
| Subset | Model | Pose ATE ↓ | Acceptance Rate ↑ | Depth AbsRel ↓ | Depth δ1 ↑ |
|---|---|---|---|---|---|
| Real | MapAnything360 | 0.134 | 94.0% | 0.070 | 46.31% |
| Argus (Ours) | 0.096 | 98.2% | 0.050 | 66.28% | |
| Synthetic | MapAnything360 | 0.087 | 99.3% | 0.063 | 74.64% |
| Argus (Ours) | 0.027 | 99.8% | 0.035 | 91.24% |
Across real residential scenes, Argus achieves an absolute camera pose error (ATE) reduction of approximately 28% and improves strict depth accuracy (δ1) by nearly 20 percentage points. In point cloud metrics, Argus reduces mean accuracy error by 37% on real scenes and 49% on synthetic scenes.
With 1.31 billion parameters, Argus reconstructs a set of 16 panoramic images in approximately 0.66 seconds using BF16 on a single H20 GPU.
Presenting at ECCV 2026 in Malmö
ECCV 2026 convened from September 8 to 12 across Malmö Arena and Malmömässan in Sweden. During the main conference poster sessions at board 007, our team connected with researchers and industry specialists exploring multi-view geometry, neural representations, and spatial AI.
Looking Ahead & Academic Collaboration
While Argus establishes strong metric fidelity within structured indoor scenes, scaling to unbounded outdoor landscapes and thousands of simultaneous views remains an active research trajectory.
We welcome joint research initiatives spanning panoramic 3D reconstruction, indoor scene understanding, multimodal foundation models, and embodied AI:
- General inquiries & collaborations: developer@realsee.com
- Corresponding author: Dr. Cihui Pan (pancihui001@realsee.com)
- Project code & weights: GitHub Repository
- Formal publication: Springer Nature Proceedings (ECCV 2026)
