Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

IR-RGB Image Registration Using Deformation Field and Mask Loss

Version 1 : Received: 22 September 2023 / Approved: 22 September 2023 / Online: 25 September 2023 (04:18:55 CEST)

How to cite: Lee, M.; Kwon, J. IR-RGB Image Registration Using Deformation Field and Mask Loss. Preprints 2023, 2023091577. https://doi.org/10.20944/preprints202309.1577.v1 Lee, M.; Kwon, J. IR-RGB Image Registration Using Deformation Field and Mask Loss. Preprints 2023, 2023091577. https://doi.org/10.20944/preprints202309.1577.v1

Abstract

This study proposes a method for image matching between infrared(IR)-RGB images using a deep learning network to estimate the deformation field. We propose a deformation field generator (DFG) that estimates the deformation field of the transformation matrix to match each pixel or IR image to the RGB image. DFG is a network that receives IR and RGB images as input; the output size is two channels and has the sample resolution as the input image. By warping the IR image through a grid-sampler that warps the image according to the value of the deformation field, we can obtain a warped IR image that matches the RGB image. Additionally, to check whether the warped IR image matched the RGB image, the masking images detecting the segmentation of objects were photographed in two images. Without directly comparing IR and RGB images, we proposed mask loss that warps the IR mask image through the deformation field and grid sampler and then compares the warped IR mask image with the RGB mask image. Mask loss solves the spatial similarity comparison problem with multi-modality images, such as IR and RGB images, by comparing the mask image with the same modality image as the mask image.

Keywords

computer vision; deep learning; multi-modality image registration

Subject

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.