Version 1
: Received: 22 September 2023 / Approved: 22 September 2023 / Online: 25 September 2023 (04:18:55 CEST)
How to cite:
Lee, M.; Kwon, J. IR-RGB Image Registration Using Deformation Field and Mask Loss. Preprints2023, 2023091577. https://doi.org/10.20944/preprints202309.1577.v1
Lee, M.; Kwon, J. IR-RGB Image Registration Using Deformation Field and Mask Loss. Preprints 2023, 2023091577. https://doi.org/10.20944/preprints202309.1577.v1
Lee, M.; Kwon, J. IR-RGB Image Registration Using Deformation Field and Mask Loss. Preprints2023, 2023091577. https://doi.org/10.20944/preprints202309.1577.v1
APA Style
Lee, M., & Kwon, J. (2023). IR-RGB Image Registration Using Deformation Field and Mask Loss. Preprints. https://doi.org/10.20944/preprints202309.1577.v1
Chicago/Turabian Style
Lee, M. and Jangwoo Kwon. 2023 "IR-RGB Image Registration Using Deformation Field and Mask Loss" Preprints. https://doi.org/10.20944/preprints202309.1577.v1
Abstract
This study proposes a method for image matching between infrared(IR)-RGB images using a deep learning network to estimate the deformation field. We propose a deformation field generator (DFG) that estimates the deformation field of the transformation matrix to match each pixel or IR image to the RGB image. DFG is a network that receives IR and RGB images as input; the output size is two channels and has the sample resolution as the input image. By warping the IR image through a grid-sampler that warps the image according to the value of the deformation field, we can obtain a warped IR image that matches the RGB image. Additionally, to check whether the warped IR image matched the RGB image, the masking images detecting the segmentation of objects were photographed in two images. Without directly comparing IR and RGB images, we proposed mask loss that warps the IR mask image through the deformation field and grid sampler and then compares the warped IR mask image with the RGB mask image. Mask loss solves the spatial similarity comparison problem with multi-modality images, such as IR and RGB images, by comparing the mask image with the same modality image as the mask image.
Keywords
computer vision; deep learning; multi-modality image registration
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
