NoMeFormer: Non-Manifold Mesh Transformer

Semantic segmentation of textured 3D meshes, i.e. the assignment of a class label to each triangle of such a mesh, is an important task in various fields. Existing deep learning models face problems when processing meshes with non-manifold structures. Most methods for 3D mesh classification rely on the assumption of manifold structure, which limits their applicability in real-world scenarios. To address this limitation, we propose NoMeFormer, a transformer-based framework specifically designed to handle any type of 3D mesh without imposing structural constraints, making it particularly suited for non-manifold mesh segmentation. A key innovation in our approach is the introduction of Local-Global (L-G) transformer blocks, which address the quadratic complexity of transformers. Initially, features are aggregated within spatial clusters of faces, followed by capturing long-range dependencies between faces via global attention. This architecture enables the model to effectively leverage both low- and high-frequency contextual information. Our experiments show that a variant of NoMeFormer based on geometrical features achieves a mean F1 score of 58.9% on the Hessigheim 3D benchmark dataset. Our framework overcomes the limitations of manifold-based approaches, offering a robust solution for semantic segmentation on non-manifold 3D meshes.