Kim, J.-K.; Jang, Y.-W.; Lee, S.; Hwang, E.-S.; Seo, Y.-H. Temporal Estimation of Non-Rigid Dynamic Human Point Cloud Sequence Using 3D Skeleton-Based Deformation for Compression. Sensors2023, 23, 7163.
Kim, J.-K.; Jang, Y.-W.; Lee, S.; Hwang, E.-S.; Seo, Y.-H. Temporal Estimation of Non-Rigid Dynamic Human Point Cloud Sequence Using 3D Skeleton-Based Deformation for Compression. Sensors 2023, 23, 7163.
Kim, J.-K.; Jang, Y.-W.; Lee, S.; Hwang, E.-S.; Seo, Y.-H. Temporal Estimation of Non-Rigid Dynamic Human Point Cloud Sequence Using 3D Skeleton-Based Deformation for Compression. Sensors2023, 23, 7163.
Kim, J.-K.; Jang, Y.-W.; Lee, S.; Hwang, E.-S.; Seo, Y.-H. Temporal Estimation of Non-Rigid Dynamic Human Point Cloud Sequence Using 3D Skeleton-Based Deformation for Compression. Sensors 2023, 23, 7163.
Abstract
This paper proposes an algorithm for transmitting and reconstructing the estimated point cloud using temporally estimating a dynamic point cloud sequence. When a non-rigid 3D point cloud sequence (PCS) is input, the sequence is divided into groups of point cloud frames (PCF), and a key PCF is selected. After the 3D skeleton is predicted through 3D pose estimation, the motion of the skeleton is estimated by analyzing the joints and bones of the 3D skeleton. For the deformation of the non-rigid human PC, the 3D PC model is transformed into a mesh model, and the key PCF is rigged using the 3D skeleton. After deforming the key PCF into the target PCF utilizing the motion vector of the estimated skeleton, the residual PC between the motion compensation PCF and the target PCF is generated. If there is a key PCF, the motion vector of the target PCF, and a residual PC, the target PCF can be reconstructed. Just as compression is performed using pixel correlation between frames in a 2D video, this paper compresses 3D PCF by estimating the non-rigid 3D motion of a 3D object in a 3D PC. The proposed algorithm can be regarded as an extension of the 2D motion estimation of a rigid local region in a 2D plane to the 3D motion estimation of a non-rigid object (human) in 3D space. Experimental results show that the proposed method can successfully compress 3D PC sequences. If it is used together with a PC compression technique such as MPEG PCC (point cloud compression) in the future, a system with high compression efficiency may be configured.
Keywords
dynamic point cloud; augmented reality; virtual reality; pose estimation; 3d skeleton; deformation; temporal prediction
Subject
Computer Science and Mathematics, Computer Vision and Graphics
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
