OpenSGA: Efficient 3D Scene Graph Alignment in the Open World
1TU Delft 2TU Munich 3ETH Zurich
Abstract
Scene graph alignment establishes object correspondences between two 3D scene graphs constructed from partially overlapping observations. It enables robots to associate observations across time and agents while preserving consistent object identities despite viewpoint changes and partial visibility. Existing approaches mainly focus on subscan-to-subscan (S2S) alignment and depend heavily on geometric point-cloud features, leaving frame-to-scan (F2S) alignment and open-set vision–language features underexplored. In addition, existing datasets for scene graph alignment remain small-scale with limited object diversity, constraining systematic training and evaluation. We present a unified and efficient alignment framework that predicts object correspondences by fusing vision–language, textual, and geometric features with spatial context. Our method introduces a distance-gated spatial attention encoder, a matching score predictor with lightweight and high-performance variants, a minimum cost flow based allocator, and a global scene embedding generator for accurate correspondence prediction even under large coordinate discrepancies. We further introduce ScanNet-SG, a large-scale dataset generated via an automated annotation pipeline, containing over 700k samples and spanning 500–3k object classes under different vocabulary settings. Experiments show that our method achieves the best overall performance on both F2S and S2S tasks, substantially outperforming existing scene graph alignment methods.
In Figures above, a result of our high-performance version model to illustrate the dataset and tasks. Below we show a flow chart of our method.
Our main **contributions** include:
(1) A unified framework for efficient scene graph alignment: We introduce an end-to-end alignment framework that performs node matching and submap-level matching, enabling robust alignment across partial and cross-view observations. We release both lightweight and high-performance versions to support different efficiency--accuracy trade-offs.
(2) Robust scene graph alignment network: We propose a distance-gated spatial attention encoder that encodes node context even under large coordinate discrepancies, together with a minimum-cost-flow-based many-to-one allocator for efficient and globally consistent matching.
(3) A scalable annotation pipeline and a large-scale dataset: We develop an automated annotation pipeline and construct ScanNet-SG, a dataset that supports both F2S and S2S alignment tasks. ScanNet-SG covers over 500 and 3k object classes (in different groups) and provides more than 700k samples in total.
Dataset
We build 3D scene graphs for the ScanNet dataset with object tags given by ScanNet human annotations and ChatGPT4o, respectively, in two groups. Blue lines show the edges connected by a distance threshold (< 2m). The image above shows scene graphs of six scenes built using tags from human annotations. Below we present another three scene graphs built using tags from ChatGPT4o. Object classes are much richer in the group using ChatGPT4o but could contain errors. We annotate 806 scenes in total using human annotated tags and 110 scenes in total using ChatGPT4o annotated tags.
Results
We present some qualitative results in the following. These results show that our method achieves the best performance in both F2S and S2S tasks. More quantitative results can be found in the paper.
Visualization of matching results produced by different models (A–E). In each subplot, the central point cloud corresponds to the full scan of Scene0673_00. The query frames are sampled from a separate scan of the same scene, Scene0673_01 (cross-scan), and are shown in the top-left as instance-segmented RGB images labeled 1–7, corresponding to ScanNet frame IDs 15, 105, 252, 639, 1050, 1521, and 1830. Objects are visualized with distinct colors, and each object center is marked with a blue dot (unmatched) or a red dot (matched). Correct correspondences are indicated by green lines, while incorrect correspondences are shown in red. Orange lines denote the ground-truth corrected correspondences for either incorrect matches or missed matches. For visualization purposes, each frame point cloud (originally in its camera coordinate system) is transformed into the world coordinate system of Scene0673_00 and then translated to a nearby location around the scan point cloud. This transformation is applied only for visualization and is not provided to any of the matching models.
Comparison of different methods on the S2S alignment task. The subscan pairs in (A–D) are: (A) Scene0673_00 frames 1383–2139 and Scene0673_01 frames 1584–1989; (B) Scene0673_00 frames 654–1140 and Scene0673_02 frames 234–1032; (C) Scene0603_00 frames 315–753 and Scene0603_01 frames 690–1317; and (D) Scene0642_00 frames 168–777 and Scene0642_01 frames 1890–2367. The overlap ratio between the two subscans decreases from 0.926 to 0.142.
Related Publications
Flow Policies as Actions of Skill-Level World Models: Learned and Symbolic Abstractions for Long-Horizon Planning
In arXiv preprint, 2026. Under review.
@misc{matoses2026flowskillwm,
title = {Flow Policies as Actions of Skill-Level World Models: Learned and Symbolic Abstractions for Long-Horizon Planning},
author = {Matoses Gimenez, Andreu and Papuc, Andrei-Carlo and Pek, Chris and Alonso-Mora, Javier},
year = {2026},
eprint = {2610.04767},
archiveprefix = {arXiv},
primaryclass = {cs.RO},
url = {https://arxiv.org/abs/2610.04767}
}
Learning Task and Motion Plans from Real Demonstrations with Hybrid Flow Matching
In arXiv preprint, 2026. Under review.
@misc{redondo2026hybridflow,
title = {Learning Task and Motion Plans from Real Demonstrations with Hybrid Flow Matching},
author = {Redondo Garcia, Zuleika and Matoses Gimenez, Andreu and Alonso-Mora, Javier},
year = {2026},
eprint = {2610.04771},
archiveprefix = {arXiv},
primaryclass = {cs.RO},
url = {https://arxiv.org/abs/2610.04771}
}
Building Explicit World Model for Zero-Shot Open-World Object Manipulation
In arXiv, 2026.
BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion
In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026.
MobileOcc: A Human-Aware Semantic Occupancy Dataset for Mobile Robots
In European Conference on Computer Vision (ECCV), 2026.
Cross-Entropy Optimization of Physically Grounded Task and Motion Plans
In IEEE Robotics and Automation Letters (RA-L), 2026.
SADCHER: Scheduling using Attention-based Dynamic Coalitions of Heterogeneous Robots in Real-Time
In IEEE Int. Symposium on Multi-Robot & Multi-Agent Systems (MRS), 2025.
Scenario-based motion planning with bounded probability of collision
In The International Journal of Robotics Research (IJRR), 2025.
TamedPUMA: safe and stable imitation learning with geometric fabrics
In Learning for Dynamics and Control (L4DC), 2025.
SHINE: Social Homology Identification for Navigation in Crowded Environments
In International Journal of Robotics Research (IJRR), 2025.
Overcoming Explicit Environment Representations with Geometric Fabrics
In IEEE Robotics and Automation Letters (RA-L), 2025.
Pushing Through Clutter With Movability Awareness of Blocking Obstacles
In IEEE Int. Conf. on Robotics and Automation (ICRA), 2025.
Globally-Guided Geometric Fabrics for Reactive Mobile Manipulation in Dynamic Environments
In IEEE Robotics and Automation Letters (RA-L), 2025.
Particle-based Instance-aware Semantic Occupancy Mapping in Dynamic Environments
In IEEE Transactions on Robotics (T-RO), 2025.
Evaluating Dynamic Environment Difficulty for Obstacle Avoidance Benchmarking
In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.
Topology-Driven Parallel Trajectory Optimization in Dynamic Environments
In IEEE Transactions on Robotics (T-RO), 2024.
Auto-Encoding Bayesian Inverse Games
In 16th Workshop on the Algorithmic Foundations of Robotics (WAFR), 2024.
Safe and stable motion primitives via imitation learning and geometric fabrics
In Robotics: Science and Systems, Workshop on Structural Priors as Inductive Biases for Learning Robot Dynamics, 2024.
Physically Grounded Optimal Realizations of Symbolic Plans
In Robotics: Science and Systems (RSS), Workshop on Frontiers of Optimization for Robotics, 2024.
Reactive grasp and motion planning for adaptive mobile manipulation among obstacles
In Robotics: Science and Systems, Workshop on Frontiers of Optimization for Robotics, 2024.
Multi-Modal MPPI and Active Inference for Reactive Task and Motion Planning
In IEEE Robotics and Automation Letters (RA-L), 2024.
Decentralized Multi-Agent Trajectory Planning in Dynamic Environments with Spatiotemporal Occupancy Grid Maps
In IEEE Int. Conf. on Robotics and Automation (ICRA), 2024.
Contingency Games for Multi-Agent Interaction
In Robotics and Automation Letters (RA-L), 2024.
Multi-Robot Local Motion Planning Using Dynamic Optimization Fabrics
In Proc. IEEE International Symposium on Multi-Robot and Multi-Agent Systems, 2023.
Learning to Play Trajectory Games Against Opponents with Unknown Objectives
In IEEE Robotics and Automation Letters (RA-L), 2023.
RAST: Risk-Aware Spatio-Temporal Safety Corridors for MAV Navigation in Dynamic Uncertain Environments
In IEEE Robotics and Automation Letters (RA-L), 2023.