Segment Anything Model for Scan-to-Structural Analysis in Cultural Heritage

Segment anything model (SAM) proposed by META Artificial Intelligence (AI) has disrupted the space of Deep Learning (DL) and Machine Learning (ML) and claims performance that outrivals conventional Convolutional Neural Networks (CNNs). The well recognized success of foundation models paved the way towards Natural Language Processing (NLP), which in computer vision domain, rests upon the concept of vision transformers (ViT). The claimed scalability and zero-shot predictability of the proposed mode has spurred extensive research among different fields. In this sense, this paper aims to investigate Segment Anything Model (SAM) potential in cultural heritage (CH) inclined scenarios. This is part of an ongoing project that seeks to implement scan-to-structural analysis at stone level. Based on the findings of this research, SAM zero-shot segmentation ability is quite promising for complex scene understanding. In addition, its tendency to favor small objects may provide a unique opportunity for damage detection and defects classification. Finally, even without any pre-training, SAM exhibits remarkable performance in edge detection and masking, without sacrificing generalizability and its accuracy in segmentation correlates to spatial resolution and background separation.