Progressive Piecewise Implicit Reconstruction from Point Cloud via Surface Accuracy Optimization

1. Introduction

Surface reconstruction is a fundamental and challenging task in the area of computer graphics or 3d computer vision, which can be beneficial to the fields of computer-aided design, scene understanding, visual navigation and robotics [1,2,3]. With the popularity of 3d scanners and depth cameras, discrete point cloud scanning data have been widely exploited as a dominant representation of complex 3d shapes or indoor scenes because of its easy acquisition and also need not require maintenance of complex topological connectivity [4,5,6,7,8]. One of the key issues is how to efficiently reconstruct a discrete surface from sparse point cloud data. Traditional geometry-based reconstruction approaches, such as Poisson surface reconstruction [9] and radial basis function based reconstruction [10] always require a high quality of model data, which will lead to significant degradation of reconstruction performance when the input data contains a lot of model noise [11,12]. Therefore, implicit reconstruction methods based on deep learning are currently gaining widespread attention [13,14,15,16], which can always be less affected by model noise and also have higher reconstruction performance.

Owing to its effectiveness in representing surfaces with arbitrarily complex topology by learning a continuous implicit field [17,18], a deep neural network based scheme can directly learn the signed distance fields (SDFs) or binary occupancy fields (BOFs) from raw point cloud data and always has satisfied generalization performance. The traditional DeepSDF method [19] learned a continuous signed distance function to represent a category of shapes via an auto-decoder, which can optimize the latent vectors and decoder weights through network back-propagation. The decoder thus can decode these latent vectors to obtain signed distance values of sampling points. The occupancy network (ONet) [20] employed the PointNet encoder [21] to encode the whole input of point cloud data thus to extract its latent feature, and then adopted a decoder to get the occupancy values of each query sampling point. Jin et al.[22] presented a neural implicit representation MuSic-UDF to leverage multi-scale dynamic grids for high-fidelity and flexible surface reconstruction from raw point clouds with arbitrary typologies. These traditional approaches essentially use deep neural networks to learn the overall shape structures but without taking into account the effective representation and learning of local shape details, thus leading to some artificial defects. Due to the voxel representation of point cloud data, Peng et al. [23] presented a convolutional occupancy network (CONet) to perform the convolutional operation on voxel grids for feature learning. Ma et al. [24] proposed Neural-Pull to pull query 3D locations to their closest points on the underlying surface using the predicted SDFs and also the gradient at the query locations. Due to its effectiveness of surface reconstruction from discrete point sets by using implicit moving least-squares (IMLS) algorithm, Liu et al. [25] incorporated IMLS surface generation into deep neural networks for inheriting both the flexibility of point sets and the high-quality of implicit surfaces. Wang et al. [26] introduced a dual octree graph network to learn adaptive volumetric shape representation for 3d shape reconstruction, which employed an encoder-decoder structure to learn feature volume based on the graph convolutions over the dual graph of octree nodes. Ma et al. [27] proposed an OSP method to reconstruct accurate surfaces from sparse point clouds with an on-surface prior, which can learn the SDFs via projecting queries onto the surface represented by a discrete point cloud data. By fusing data from multiple features, Le et al. [28] devised a multifeature data fusion model (MfDFM) that enhances measurement data with high-frequency geometric information to achieve highly accurate surface reconstruction. Chu et al. [29] introduced a diffusion-based generative 3d shape completion method by casting shape completion as a generative task conditioned on the incomplete shape. Ren et al. [30] employed a hierarchical voxel latent diffusion model XCube to generate progressively the higher resolution grids in a coarse-to-fine manner. Yin and Lv [31] presented a self-tuned two-stage framework for point cloud reconstruction, which combines a parameter-free denoising module (TPDn) for noise or outliers removal and thus upsampled by PU-Net to recover fine-grained geometry. By integrating a divide-and-conquer strategy into SDF-based reconstruction, Yang et al. [32] proposed a scalable and high-quality neural implicit representation for 3d scene reconstruction. Based on RGB-D inputs, Jiang et al. [33] presented a radiance field rendering method for indoor scenes reconstruction without pre-computed camera poses. Cao et al. [34] introduced PointAMaLR to enhance feature representation and processing accuracy through attention-guided multi-scale local reconstruction. However, these methods always require voxelization of the whole 3d embedding space, resulting in an exponential increase in the amount of input voxel data and the size of network parameters. So, it is difficult to reconstruct high-resolution 3d complex shapes due to its heavy network training and significant memory demands.

Based on a multi-scale neighboring point feature aggregation module, our presented network efficiently and robustly learns input point cloud features. Moreover, through a surface accuracy optimization (SAO) strategy, the proposed novel piecewise implicit surface reconstruction network progressively learns the implicit surface while controlling its accuracy under a predefined tolerance parameter—starting with coarse overall shapes and advancing in a coarse-to-fine manner to capture increasingly fine local geometric details (see Figure 1). The main contributions of this work can be summarized as follows:

• A piecewise surface reconstruction scheme is presented for implicit surface learning, which can effectively train the local signed distance field of point cloud shapes.
• A multi-scale neighbouring point feature aggregation module is introduced to effectively extract the local features of sampling points, and a decoder is designed that adopts a fully connected residual feature decoding for effectively learning the context information of surface patches.
• Owing to the SAO strategy, the accuracy of implicit surface learning can be controlled by a tolerance parameter, and thus can reconstruct local geometry details of point cloud data progressively in a coarse-to-fine manner.

2. Methods

Owing to the SAO strategy, our proposed network takes discrete point cloud data as input directly and trains the local signed distance field of shape patches. The overall network framework is shown in Figure 2.

The proposed reconstruction network can be divided into the following steps:

• A farthest point sampling algorithm [35] is used to obtain the initial centre and radius of the spherical neighbourhood points to generate the initial surface patches. Then the offset of each sample point within each patch relative to its centre can be calculated separately as the relative coordinates, where the discrete points located in the overlap regions may have several relative coordinates.
• In the network coding stage, two neighbouring point feature aggregation modules of different scales are employed to encode the relative coordinate of each sample point into a 256-dimensional latent vector, thus extracting the feature information of their adjacent sampling points. The first module aggregates 8 neighbouring points around each sample point and the other one is 16. Then the multi-scale features of sampling points can be obtained by stitching these feature information introduced by two modules together, and the corresponding latent vectors of sampling points can be obtained by 4-layer MLP perceptron $(512,512,256,256)$. These latent vectors of sampling points located on each surface patch can be maximized to characterize the latent features of the surface patch.
• In the network decoding stage, according to the surface optimization strategy, a tolerance parameter is introduced to control the reconstruction accuracy of our network learning. At the beginning of network training, only a shallow network is adopted and the number of network layers can be gradually increased as the number of training rounds. During network learning, the relative coordinates and their corresponding latent features of surface patches will be input to the network decoder to obtain the signed distances of the relative positions of each sampling point. The signed distances of the relative positions of sampling points located in the overlapping regions of different surface patches are weighted and summed to obtain the signed distances of sampling points.
• Based on the Marching Cube algorithm [36], we can extract the zero iso-surface of the signed distance fields to obtain a final 3d mesh model.

2.1. Surface Accuracy Optimization Strategy

Inspired by the human learning mechanism, organizing the tasks for shape feature learning in a simple-to-complex manner can effectively improve the reconstruction performance of our network [37]. In this paper, we adopt a supervised learning strategy to efficiently learn the implicit signed distance field, making the predicted SDF values of sampled points as close as possible to their true SDF values. Considering its effectiveness of supervision with the true SDF values, our proposed reconstruction network will optimize the surface accuracy-based strategy to better learn the local features of point cloud shapes. Specifically, instead of strictly requiring the reconstruction network to estimate SDF values as close as possible to the true SDF values, the accuracy of the learned surface is controlled by an upper tolerance parameter $ϵ$ for surface reconstruction error, which allows the network to estimate the underlying shapes with small errors, i.e., the estimated SDF values with errors smaller than $ϵ$ are considered to be correct.

For our experiments, start with the relatively large parameter $ϵ$ which can focus on the global structure recovery of point cloud shape. Then, gradually lower $ϵ$ will expose more fine details of the shape until it drops to $0.0$. As shown in Figure 3, there is an error region of thickness $2\ast ϵ$ near the exact surface of the point cloud shape, and the goal of the reconstruction network is to learn and reconstruct shape structures located in this local region. At the beginning of network training, a relatively large value of $ϵ$ is set to allow the network to learn relatively smooth and coarse surfaces in this local region. The value of $ϵ$ is gradually reduced to expose and learn more fine shape details as the number of training rounds increases, and eventually, the value of $ϵ$ is set to $0.0$ to predict the exact reconstructed shape. So, the loss function for surface reconstruction can be designed as follows,

$$L_{recon}=\sum _{\mathit{p}\in \left|S\right|}max\left\{\right|f\left(\mathit{p}\right)-g\left(\mathit{p}\right)|-ϵ,0\}$$

where $\left|S\right|$ is the set of sampled points, $f\left(\mathit{p}\right)$ means the estimated SDF value of the sampling point, and $g\left(\mathit{p}\right)$ is the corresponding true SDF value of the sampled point.

2.2. Calculation of Signed Distance Values for Sampling Points

For effectively learning the local shape features using a piecewise surface manner, the input discrete point cloud data is divided into surface patches, and the discrete point cloud within each patch is regarded as a local region where the neighbouring information of the surface patch can be represented as a spherical neighbourhood with the patch centre at $\mathit{c}$ and the radius of r. Before encoding the input 3d point cloud data, the initial surface patch needs to be generated efficiently so that each surface patch will contain the sampling points as completely and uniformly as possible. Specifically, the input point cloud is first sampled using the farthest point sampling algorithm [35] to obtain the representative point cloud $\Pi \in R^{K\times 3}$, (where K is the number of sampling points located in the representative point cloud and is taken as 30 for our experiments). The algorithm is performed as follows: firstly, we select a random seed point $\mathit{s}$ from the input point cloud data and add it to $\Pi$, then we calculate the Euclidean distance between the seed point $\mathit{s}$ and the remaining input sampling points. Secondly, we select the farthest sampling point to be added to $\Pi$, and so on until the number of sampling points included in $\Pi$ is K; and the 3d coordinates of each sampling point of $\Pi$ are then selected as the centre of each surface patch, i.e.

$${\mathit{c}}_i={\mathit{s}}_i\in \Pi ,i=0,1,2,...,K$$

Furthermore, to determine the radius of each surface patch, we select a random sampling point $\mathit{p}$ from the input point cloud data and calculate the Euclidean distance between all sampling points in $\Pi$ and find the closest sampling point $\mathit{s}$, which we can mark $\mathit{p}$ as a reference point for sampling point $\mathit{s}$. And so on until all the input sampling points are marked as reference points, and thus we calculate the radius of each surface patch as follows,

$$r_i=ma{x}_{\mathit{p}\in T_{{\mathit{s}}_i}}{\parallel {\mathit{s}}_i-\mathit{p}\parallel }_2,i=0,1,2...,K$$

where $T_{{\mathit{s}}_i}$ indicates all the reference points of sampling point ${\mathit{s}}_i$.

To effectively learn the local feature differences of sampling points, instead of taking the original coordinates of sampling points as the input of the encoder, here we employ the relative coordinates of sampling points, that is, the offset of each sample point within each patch relative to its centre respectively, where the discrete points located in the overlap regions may have several relative coordinates. Then, as a pre-processing step before point cloud encoding, the weights of sampling points in each surface patch can be determined according to their Euclidean distances between sampling points and the centre of the surface patch as follows,

$$w_i\left(\mathit{p}\right)=exp(-{\displaystyle \frac{\parallel {\mathit{c}}_i{-\mathit{p}\parallel }^2}{2{\sigma }^2}})-exp(-{\displaystyle \frac{r_i^2}{2{\sigma }^2}})$$

Here $w_i\left(\mathit{p}\right)$ means the weight of sampling point $\mathit{p}$ as to the i-th surface patch $Patc{h}_i$. If the sampling point is not located within the i-th surface patch, the weight $w_i\left(\mathit{p}\right)$ is zero, otherwise the smaller the distance between $\mathit{p}$ and centre of i-th surface patch $Patc{h}_i$, the larger the weight. Here $\sigma$ is taken as ${\displaystyle \frac{r_i}{3}}$.

Further, to extract and aggregate local information of adjacent sampling points, we encode the relative coordinates of each sample point in our network encoder and generate a 256-dimensional latent vector by using the neighbouring point aggregation strategy. Meanwhile, the latent vector $\mathit{z}$ of the surface patch can be created as the maximum of the latent vectors of sampling points located in this surface patch. Finally, during the network decoding step, a multi-layer fully connected residual feature decoding module is employed to obtain the signed distance values of each sample point. It should be noted that for sampling points that are located in the overlap regions of several adjacent patches, the signed distance of sampling points could be weighted and the sum of the signed distance values of adjacent patches as follows,

$$f\left(\mathit{p}\right)=\sum _i{w}_i\left(\mathit{p}\right)f_i\left(\mathit{p}\right)/\sum _i{w}_i\left(\mathit{p}\right)$$

2.3. Encoder via Multi-Scale Neighboring Point Feature Aggregation

For point cloud surface reconstruction, due to the inconsistency of sampling density in different regions of point cloud shape, capturing the point feature information only using a single scale neighbouring points may not be able to learn the local structure of the underlying 3d shapes due to its sparse sampling points in that region. To solve this issue, we adopted the neighbouring point feature aggregation module [38] to fully extract the multi-scale point feature information during the network learning, which can better capture the local shape structure of the underlying point cloud shape.

As shown in Figure 4, taking the 3d coordinates of sampling points as the input of network encoder, and the $N\times 128$ dimensional point features are first obtained by using a 2-layer perceptron $(64,128)$; then, the multi-scale point features can be extracted by two different scales of the neighbouring point aggregation module and thus stitched then together. As shown in Figure 4, for each neighborhing point feature aggregation module, the local point feature information is firstly extracted by two branches: the first branch aggregates M neighboring points around each sampling point using k-NN algorithm to obtain $N\times M\times 128$ dimensional feature matrix ${\mathit{P}}_1$; the second branch expands the local feature information of by a factor M of to obtain the $N\times M\times 128$ dimensional feature matrix ${\mathit{P}}_2$; then these two feature matrices are subtracted to obtain a $N\times M\times 128$ dimensional feature matrix ${\mathit{P}}_3$; and the feature matrix ${\mathit{P}}_2$ and ${\mathit{P}}_3$ are then concatenated to obtain the $N\times M\times 256$ dimensional feature matrix, and finally the maximum pooling operation is performed on the second dimension of feature matrix to obtain the $N\times 256$ dimensional neighbouring feature aggregation information of point cloud data. The second point feature aggregation module also performs the above neighbouring point feature aggregation step to form a $N\times 256$ dimensional neighbourhood feature aggregation information of the point cloud. We can thus stitch these two single-scale point features from the above two branches using concat operation to obtain $N\times 512$ dimensional multi-scale point feature information. Finally, the $N\times 256$ dimensional encoding results of the neighbouring aggregation features of the whole point cloud shape can be obtained using a stacking 4-layer perceptron $(512,512,256,256)$. In our experiments, these two different scales are set to 8 and 16 respectively, which can demonstrate that our strategy of extracting and combining different scale point features is always superior to that of only using a single scale feature encoding in terms of point feature learning.

2.4. Decoder via Surface Accuracy Optimization

During the network encoder encoding the aggregated point features, to effectively estimate the signed distance values of discrete sampling points, both the relative coordinates of sampling points and the aggregated latent features of the corresponding surface patches are adopted as the input to our network decoder, such as shown in Figure 5. Meanwhile, we employ a SAO strategy for our decoder structure. That is, we first use a shallow network to learn a coarse 3d shape with a large value of tolerance parameter $ϵ$ at the beginning of network training, and then add more network layers to improve reconstructed surface accuracy while decreasing the value of tolerance parameter $ϵ$ as the number of training rounds increases. The number of decoder network layers, the value of discrete interpolation parameter $ϵ$, and the number of training rounds are listed in Table 1.

However, the trained shallow networks may be affected due to the addition of a new untrained layer to our network. Therefore, after adding a new layer, we also give this new layer a weight of $\alpha$, which corresponds to the weight of the old layer of $1.0-\alpha$. Here, the weight of $\alpha$ can linearly increase from $0.0$ to $1.0$ according to the number of training rounds, so that the original network can smoothly adapt to this new layer. The output of the proposed network in this newly added layer can be calculated as follows.

$${\mathit{F}}_w={\mathit{F}}_o\times (1.0-\alpha )+{\mathit{F}}_n\times \alpha (\alpha =0.001\times epoc{h}_n)$$

where ${\mathit{F}}_o$ denotes the feature matrix of the previous old layer, ${\mathit{F}}_n$ means the feature matrix of ${\mathit{F}}_o$ by using the new layer, ${\mathit{F}}_w$ is the feature matrix of the final network output at this new layer, $epoc{h}_n$ is the current number of training rounds. In addition, by considering the issue of network degradation due to the large number of decoder layers, we also add the residual network structure [39] to the shallow part of our decoder to improve the network performance. As shown in Figure 5, the shallow network of the decoder includes four fully-connected residual feature decoding modules. Specifically, let the input feature matrix of the module is ${\mathit{F}}_0$, and ${\mathit{F}}_1,{\mathit{F}}_2,{\mathit{F}}_3$ denote the output of the 1-st, 2-nd, 3-th fully-connected layer respectively, and $\mathit{W}$ means the weight of the neurons including in fully-connected layer, then the output of each network layer of the module can be calculated as follows,

$$\begin{array}{c}\hfill F_1=\sigma {\mathit{W}}_{{\mathit{F}}_1}·{\mathit{F}}_0\\ \hfill F_2=\sigma {\mathit{W}}_{{\mathit{F}}_2}·{\mathit{F}}_1\\ \hfill F_3=\sigma {\mathit{W}}_{{\mathit{F}}_3}·{\mathit{F}}_2\end{array}$$

Here, $\sigma$ denotes the ReLU activation function. The final output of the module can be expressed as follows,

$${\mathit{F}}_4=\sigma {\mathit{F}}_3+{\mathit{F}}_1$$

where + denotes matrix summation. It should be mentioned that the information loss can be effectively reduced during feature propagation by adding the residual structure, which can enhance the gradient propagation during network training and also improve the decoder performance of decoding features.

2.5. Loss Function

During our task of piecewise surface reconstruction, to control the effective generation of surface patches and learning of the shape latent features [40], the following four items of loss functions are considered for the network training.

• $L_{sur}$: Keeps surface patches of the same object as close to the object surface as possible,

  $$L_{sur}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{i=0}^{B-1}}{\displaystyle \sum _{j=0}^{K-1}}max\{mi{n}_{\mathit{p}\in O_i}\parallel {\mathit{c}}_{i,j}-\mathit{p}{\parallel }_2^2,T\}$$

  where $O_i$ is the set of all sampling points located on the i-th object and B is the number of objects in a training batch during network training. The loss function is only used when the distance between the object surface and the patches is greater than a threshold T. Since we eventually need to extract the zero iso-surface of sampling points, the centres of each surface patch should be as close as possible to the 3d shape for the selection of sampling points thus using the SAO strategy. The introduced threshold T should be taken as a small value, so we choose $0.06$ in our experiments.
• $L_{cov}$: Makes the sampling points should be located within at least one surface patch,

  $$L_{cov}={\displaystyle \sum _{i=0}^{B-1}}{\displaystyle \frac{1}{|U_i|}}\sum _{\mathit{p}\in U_i}{\displaystyle \frac{w_{i,j,\mathit{p}}\left(\parallel {\mathit{c}}_{i,j}{-\mathit{p}\parallel }_2-r_{i,j}\right)}{{\sum }_j{w}_{i,j,\mathit{p}}}}$$

  where $U_i\in O_i$ is the set of all sampling points for ith-objects that do not located in any of surface patches, and $w_{i,j,\mathit{p}}$ means the weight of sampled points $\mathit{p}$ sampled from ith-object that located in the jth-patch.
• $L_{scl}$: Keeps the surface patch as small as possible to prevent significant overlap between different patches,

  $$L_{scl}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{i=0}^{B-1}}{\displaystyle \sum _{j=0}^{K-1}}{r}_{i,j}^2$$
• $L_{var}$: Makes the size of different patches similar to prevent the surface reconstruction only using few large patches,

  $$L_{var}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{i=0}^{B-1}}{\displaystyle \sum _{j=0}^{K-1}}{(r_{i,j}-m_i)}^2$$

  where $m_i$ is the average radius of the surface patch located in the i-th object, the larger the patch radius the larger the value of the loss function, thus can prevent the reconstruction network from generating large surface patches.

Finally, the loss function of our network can be defined as follows,

$$L=L_{recon}+\alpha L_{sur}+\beta L_{cov}+\gamma L_{scl}+\delta L_{var}$$

Here, the weight of $L_{recon}$ is set to $1.0$ to be defined as a benchmark, and thus we can determine the weights of other losses. The other losses are designed so that as many sample points located near the object’s surface as possible are included in the surface patches. So, we always set large weights to the $L_{sur}$ and $L_{cov}$ losses and small weights to $L_{scl}$ and $L_{var}$. In our experiments, the weights of this total loss function are selected as $\alpha =5.0$, $\beta =1.0$, $\gamma =0.01$, and $\delta =0.01$.

3. Experimental Results

The proposed implicit surface reconstruction network has been trained on an NVIDIA RTX 3060 GPU using the PyTorch framework. As a preprocessing step, a total of 200,000 discrete sampling points are sampled from each 3d object. In our experiments, a total of $3,072$ randomly selected sampling points from the point cloud shapes contained in each training batch are chosen as the input to the reconstruction network and their signed distance values can be effectively learned. The presented network is trained with the help of Adam Optimiser. The learning rate is set to $0.0005$, which is halved every 200 round, and the batch size is 2.

3.1. Surface Reconstruction via Our Network

To illustrate the effectiveness of our reconstruction network, we first train and test our network on the public ShapeNet dataset [1], which is a large-scale public 3d shape dataset produced by researchers from Princeton University, Stanford University, and TTIC. We select some typical 3d shapes, such as sofa, chair, airplane, and other categories from the ShapeNet dataset, and 100 models are selected for each object category and trained for 1000 rounds. The surface reconstruction results are shown in Figure 1 and Figure 6. From these experiments, it can be seen that the introduced method can not only completely reconstruct the overall shape of a 3d object, but also effectively reconstruct the local fine structures such as the pillows on the sofa, the foot of the chair, the tail of the airplane, the back of the individual seat, and the lamp holder. In general, the reconstructed model is not only consistent with the real object in terms of overall structure but also reconstructs the complex fine structures of 3d objects and suppresses the artefacts.

Furthermore, to verify its generalization of the proposed reconstruction network, we also experimented on the Famous dataset [41] and used our network to over-fit each point cloud shape. Reconstruction results are shown in Figure 7. These experimental results demonstrate that our method can effectively reconstruct the overall structure of the objects with different shape categories from the ShapeNet dataset [1], which can validate its ability of good generalization.

3.2. Method Comparisons

To illustrate the effectiveness of our presented network, we compared our network with the existing methods and employed the same subset of models selected from the ShapeNet dataset [1] as the network training and test sets. The occupancy network [20] and convolutional occupancy network [23] use the pre-trained models provided by their authors, while the Intersection on Union (IoU) and Chamfer distance (CD) error [22] are adopted as the metrics to evaluate their effectiveness of object reconstruction.

Figure 8 gives the reconstruction results both of our network and other existing methods. As shown in Figure 8, the DeepSDF network [19] and occupancy network [20] can only reconstruct the general overall structure of the underlying point cloud shapes, and there are a lot of man-made artefacts around the reconstructed model, whilst the convolutional occupancy network [23] and our presented network can reconstruct the overall shape of point cloud model accurately (such as motorboat and rifle), and the reconstruction results are significantly better than the above two networks. In terms of shape details, our proposed network can generate fewer man-made artefacts if reconstructing sub-parts with large holes, such as the cabin of a motorboat. The DeepSDF network [19] and the occupancy network [20] always introduce more man-made artefacts, and is also difficult to recover those local fine structures. This is because the global feature learning method only can learn their overall shape, and does not pay enough attention to the local fine shape details, which leads to its difficulty in reconstructing the fine structures of point cloud shapes. Furthermore, compared with the traditional convolutional occupancy network [23], our proposed network can obtain better reconstruction results for fine structures, such as gun barrels, the helm of the ship, the leg of the sofa, and the tail tip of the aircraft, etc, which indicates that our introduced reconstruction network outperforms the traditional convolutional occupancy network [23] in terms of learning local fine structures. Moreover, the reconstruction of shape components (such as the gap between pillows, scope, and body of the gun) always leads to less adhesion between different components, which indicates that our proposed network can suppress reconstruction artefacts better than that of a convolutional occupancy network [23].

Compared with the global feature learning networks, such as DeepSDF network [19] and occupancy network [20], our proposed network can effectively learn local features of the underlying shapes, which can not only provide a completed reconstruction with fewer artefacts but also effectively reconstruct the local fine shape details. Compared to the convolutional occupancy network [23], our presented network can effectively extract the shape features by using multi-scale neighbouring point feature aggregation operation in the encoding stage, and also reduce the loss of feature information during gradient propagation through the residual structure in the decoding stage. Meanwhile, owing to the surface accuracy optimization strategy, our reconstruction network can understand and effectively learn the complex structure information of the underlying shapes, and thus can reconstruct the local shape details in a more fine-grained manner, which validates a better reconstruction performance than that of convolutional occupancy network [23].

Table 2 shows the data statistics for different reconstruction networks on the ShapeNet dataset [1]. The higher the IoU metric and the lower the CD error metric, the closer the reconstruction results are to the ground truth, which gives better reconstruction results. From the data in Table 2, it can be seen that our proposed network and the convolutional occupancy network [23] outperform traditional DeepSDF network [19] and the occupancy network [20] in terms of evaluation metrics, which demonstrate its advantage of local feature learning over global feature learning. Such as, the reconstruction results on the ShapeNet dataset of our network are improved by an average of $7.11\%$, $20.15\%$, and $1.99\%$ in terms of IoU index, comparing with DeepSDF, ONet, and CONet methods, respectively; and also reduced by an average of $76.28\%$, $83.33\%$, and $36.97\%$ in terms of CD error, respectively. Moreover, our proposed network has the best values in both evaluation metrics and also shows better reconstruction performance.

4. Discussion

To perform an ablation study, here we first construct a baseline network for surface reconstruction that only uses 10 patches without the SAO strategy and also remove the fully connected residual feature decoding module and the multiscale neighbouring point feature aggregation module. Then, we also train our network by using full 30 patches, and adding the SAO optimization strategy, and adding the other modules to our baseline network. For the network training, we select 100 point cloud models of the chair category from the ShapeNet dataset [1]. The quantitative statistics of the reconstruction are shown in Table 3. Experimental results demonstrate that the reconstruction performance of the network gradually increases with the number of surface patches and the addition of different modules. While the number of surface patches is increased from 10 to 30 in the baseline network, the reconstruction performance is significantly improved because a larger number of patches can always learn local fine feature information even in a smaller object region. On the other hand, if the surface accuracy optimization strategy is included in the reconstruction network, it can gradually learn more and more complex structures of point cloud shapes, which can easily understand the point cloud information than that of the direct feature learning and thus achieve better reconstruction performance.

5. Conclusions

Owing to a SAO strategy, we proposed a piecewise implicit surface learning network for surface reconstruction. Taking the point cloud data as direct input, the network adopted a multi-scale neighbouring point feature aggregation module to effectively extract the local feature information of sampling points. Moreover, we introduced a tolerance parameter to control the reconstruction accuracy of the piecewise reconstruction network, that is, only shallow layers are employed at the beginning of the network training, and the number of layers is gradually increased with the number of training rounds, which allows the reconstruction network to learn more and more complex shape feature progressively. Experiments illustrate that our reconstruction network can effectively reconstruct the local fine structures of 3d objects, reducing the incorrect adhesion between model components, and leading to only fewer reconstruction artifacts.

However, to better reconstruct the detailed features of 3d objects, future work could consider how to employ optimization methods such as pre-processing the original point cloud data to aid 3d reconstruction. Meanwhile, the encoder structures may be optimized and improved to speed up the network training whilst better learning the intrinsic features of point cloud shapes.