Soft Voxelizer: a differentiable voxelizer for multi-representational 3D geometry processing

Abstract

In this thesis, I propose $\textit{Soft Voxelizer}$, a differentiable voxelizer that can be used in 3D geometry processing tasks involving both meshes and voxel images. By generating softened voxelization results, Soft Voxelizer allows gradients with respect to voxels to propagate across the voxelizer, enabling mesh-side optimization from voxel-side loss functions. With a softness hyperparameter, it also provides some controls over the loss landscape for fast and optimal convergence. Using the proposed voxelizer, one can iteratively deform a base mesh to fit a given voxel image through the gradient descent. Experiments also demonstrate that a medical image segmentation network can be trained on the mesh-side with the loss defined on the voxel-side.