Self-distillation for Efficient Object-level Point Cloud Learning

The emerging accessibility of 3D point cloud data has catalyzed the evolution of deep-learning methodologies for analysis and processing of 3D data. However, the efficacy of neural networks in this domain is often inhibited by the necessity for extensively labelled datasets. In this study, we investigate the application of self-distillation techniques based on Siamese networks, BYOL and SIMSIAM, to pre-train encoders designed for 3D point cloud processing. These pre-training regimes enable encoders to generate data representations without label reliance, potentially supporting network performance in downstream tasks. The efficacy of these learned representations was assessed using the established evaluation methodologies for pre-training: linear probing and finetuning. We also incorporate an analysis of self-supervised features in a retrieval scenario. Furthermore, the impact of these representations on subsequent applications was evaluated via transfer learning by employing pre-trained models as a foundation for standard test datasets.