Nothing Stands Still: A Spatiotemporal Benchmark on 3D Point Cloud Registration Under Large Geometric and Temporal Change

Abstract

Building 3D geometric maps of man-made spaces is a well-established and active field that is fundamental to numerous computer vision and robotics applications. However, considering the continuously evolving nature of built environments, it is essential to question the capabilities of current mapping efforts in handling temporal changes. In addition to the above, the ability to create spatiotemporal maps holds significant potential for achieving sustainability and circularity goals. Existing mapping approaches focus on small changes, such as object relocation within common living spaces or self-driving car operation in outdoor spaces; all cases where the main structure of the scene remains fixed. Consequently, these approaches fail to address more radical change in the structure of the built environment, such as on the geometry and topology of it. To promote advancements on this front, we introduce the Nothing Stands Still (NSS) benchmark, which focuses on the spatiotemporal registration of 3D scenes undergoing large spatial and temporal change, ultimately creating one coherent spatiotemporal map. Specifically, the benchmark involves registering within the same coordinate system two or more partial 3D point clouds (fragments) originating from the same scene but captured from different spatiotemporal views. In addition to the standard task of pairwise registration, we assess multi-way registration of multiple fragments that belong to the same indoor environment and any temporal stage. As part of NSS, we introduce a dataset of 3D point clouds recurrently captured in large-scale building indoor environments that are under construction or renovation. The NSS benchmark presents three scenarios of increasing difficulty, with the goal to quantify the generalization ability of point cloud registration methods over space (within one building and across buildings) and time. We conduct extensive evaluations of state-of-the-art methods on NSS over all tasks and scenarios. The results demonstrate the necessity for novel methods specifically designed to handle large spatiotemporal changes. The homepage of our benchmark is at http://nothing-stands-still.com.