A Task-driven Network for Mesh Classification and Semantic Part Segmentation

Abstract

Given the rapid advancements in geometric deep-learning techniques, there has been a dedicated effort to create mesh-based convolutional operators that act as a link between irregular mesh structures and widely adopted backbone networks. Despite the numerous advantages of Convolutional Neural Networks (CNNs) over Multi-Layer Perceptrons (MLPs), mesh-oriented CNNs often require intricate network architectures to tackle irregularities of a triangular mesh. These architectures not only demand that the mesh be manifold and watertight but also impose constraints on the abundance of training samples. In this paper, we note that for specific tasks such as mesh classification and semantic part segmentation, large-scale shape features play a pivotal role. This is in contrast to the realm of shape correspondence, where a comprehensive understanding of 3D shapes necessitates considering both local and global characteristics. Inspired by this key observation, we introduce a task-driven neural network architecture that seamlessly operates in an end-to-end fashion. Our method takes as input mesh vertices equipped with the heat kernel signature (HKS) and dihedral angles between adjacent faces. Notably, we replace the conventional convolutional module, commonly found in ResNet architectures, with MLPs and incorporate Layer Normalization (LN) to facilitate layer-wise normalization. Our approach, with a seemingly straightforward network architecture, demonstrates an accuracy advantage. It exhibits a marginal 0.1% improvement in the mesh classification task and a substantial 1.8% enhancement in the mesh part segmentation task compared to state-of-the-art methodologies. Moreover, as the number of training samples decreases to 1/50 or even 1/100, the accuracy advantage of our approach becomes more pronounced. In summary, our convolution-free network is tailored for specific tasks relying on large-scale shape features and excels in the situation with a limited number of training samples, setting itself apart from state-of-the-art methodologies.

Introduction

In this paper, (1) We introduce a task-driven neural network that accommodates diverse mesh triangulations/resolutions and manifold/non-manifold mesh models. (2) Our method demonstrates robust learning capabilities for mesh classification and semantic segmentation tasks, surpassing existing state-of-the-art methods, especially in scenarios with limited data availability. (3) Extensive experiments were conducted, and all the results consistently validate the efficacy of our proposed approach.

(a) Our method showcases a noteworthy ability to effectively handle mesh models with varying resolutions. Furthermore, it exhibits robustness when confronted with different triangulations, even in cases where the input model contains elongated and narrow triangles, as exemplified by the Vase model on the right. (b) We employ the red star as the reference for visualizing the correlation with other destination points. The color bar indicates that the weakest correlation is denoted by the color magenta. Remarkably, the correlation between the red star and the yellow star persists even when they are separated by a considerable distance.

In semantic part segmentation tasks, it has been observed that the segmentation boundary (highlighted in red) tends to align with the negative minima of the principal curvatures. This alignment suggests that dihedral angles can serve as valuable indicators in semantic part segmentation.

Results

A gallery of segmentation results of the COSEG dataset. From left to right: Tele-aliens, Chairs, and Vases.

We conducted tests on the Human Body dataset. The segmentation results are visually depicted in this figure.

The segmentation results for intracranial aneurysms demonstrate the high accuracy achieved by our method.

Our method demonstrates the capability to perform segmentation on non-watertight data, as shown on the left, and non-manifold data, as depicted on the right.