Application of object detection and semantic segmentation in structure from-motion mappings
0Wilhelm Hannemann, Jessica N. Meyer
One aim of the BMBF-sponsored research project “Altbergbau 3D” is to make inaccessible parts of the World Heritage Site “Rammelsberg” visible and to make the resulting data available for transdisciplinary study and interpretation. Structure-from-Motion (SfM) photogrammetry has been used for the archaeological documentation of various historic mining sites in recent years due to its effectiveness in rough, spatially confined spaces. This method produces high-resolution 3D models alongside thousands of photos with known 3D orientation.
This paper explores the use of deep learning methods for automated object detection and semantic segmentation in 3D mining environments. The vast number of photographs makes it possible to apply 2D image-based techniques to extract relevant and irrelevant information, which can then be transferred into 3D using SfM orientation data.
Two machine learning tools are discussed: one network is trained to identify specific mining-related features (e.g., rock bolts and boreholes) across the entire site and determine their approximate 3D positions, allowing quick analysis of spatial distribution. A second network is trained on a smaller set of manually segmented photos to detect and label unwanted elements like chain link fabric at the pixel level. This network is then applied across the full dataset, enabling the masking of such elements during dense 3D point cloud computation. The goal is to produce cleaner, less obstructed views of the mine walls for more effective analysis.