An Edge-Deployable Visible-Thermal Framework for Cow-Head Detection and Cross-Camera Association in Robotic Dairy Monitoring

1. Introduction

Continuous and low-disturbance monitoring has become increasingly important for precision dairy-cow management. Body-temperature variation is closely related to thermoregulation, heat stress, inflammatory responses, disease risk, and reproductive activity, and therefore provides useful information for animal health and welfare assessment [1,2,3]. Rectal thermometry and wearable sensors can provide direct or continuous measurements, but they usually require animal contact, restraint, device attachment, or repeated close human–animal interaction. These procedures increase labour demand and may disturb normal behaviour, which is not ideal for frequent monitoring during routine barn inspection. Infrared thermography provides a non-contact alternative by recording surface thermal radiation. [4].

Several facial thermal regions have been investigated as potential physiological windows in cattle. Ear surface temperature measured using infrared thermography has been explored as a non-invasive variable for detecting postpartum hypocalcaemia in Holstein dairy cows [5]. Eye and muzzle temperatures of calves have been automatically extracted using infrared thermography and object-detection algorithms, indicating the potential of continuous facial temperature measurement for welfare-related monitoring [6]. In reproductive management, thermal infrared features from the eye and vulva have been used for non-contact oestrus monitoring in cows [7]. More broadly, automatic collection of facial temperatures from the eyes, muzzle, nostrils, ears, and horns has been reported in dairy cows, showing that multiple facial regions may provide useful thermal information for animal-health and welfare research [8].

Nevertheless, enabling an inspection robot to acquire such thermal information from multiple cows during routine barn patrol remains challenging. In multi-cow scenes, each thermal observation must be associated with the correct animal before it can support individual-level monitoring. This requirement is difficult to meet using a single infrared camera because infrared pseudo-colour images provide thermal information but often contain weak texture, blurred anatomical boundaries, and limited local structural details. These limitations make it difficult to localize cow heads and distinguish individual animals reliably, especially when cows overlap, fences occlude the head region, or the robot viewpoint changes. In contrast, visible-light images provide richer contours, textures, and structural information for cow-head localization, but they do not contain temperature information. Therefore, visible-light and thermal-infrared cameras have complementary roles: visible-light images are suitable for stable cow-head localization and target separation, whereas infrared images provide the thermal information needed for later facial-temperature extraction. The key prerequisite is then to establish target-level correspondence between cow heads detected in the two camera views, which provides a front-end basis for subsequent local registration, facial-region recognition, and radiometric temperature extraction.

For this complementary sensing strategy to be useful in robotic inspection, the same cow head must be correctly associated across synchronized visible-light and infrared image streams. If cross-camera correspondence is incorrect, the subsequent thermal information may be assigned to the wrong animal, especially in crowded scenes with occlusion, unequal target numbers, and different fields of view. A mobile robot equipped with visible-light and infrared sensors can provide a practical platform for such monitoring, but it also introduces additional challenges, including vibration, changing viewpoints, installation offsets between cameras, and limited edge-computing resources [9,10]. Therefore, an effective robotic monitoring framework requires both a lightweight cow-head detector and a target-level heterogeneous-camera association method.

Vision-based livestock perception has been widely studied for dairy-cow detection, identification, behaviour analysis, and health monitoring. Song et al. reviewed video-based methods for dairy-cow detection, individual identification, and behaviour recognition, highlighting their value for intelligent dairy management [11]. Tassinari et al. applied deep learning to detect dairy cows in free-stall barns, demonstrating the feasibility of automated perception in structured housing environments [12]. Tian et al. proposed a robot-assisted method for recognizing selective feeding behaviour in dairy cows, showing the potential of mobile robotic platforms in barn monitoring [13]. Bumbálek et al. integrated computer vision and infrared thermography for hoof-disease localization in dairy cattle, linking visual detection with thermal analysis in real farm scenes [14]. Wang et al. used computer vision to analyse multi-object cattle rumination through cow-head detection and tracking [15]. These studies provide important foundations, but many of them rely on fixed cameras, single imaging modalities, or separate visual and thermal analyses, rather than establishing target-level correspondence between visible-light and infrared images acquired by a moving robot.

Cross-modal image registration and visible–infrared association have also been investigated in broader computer-vision applications. Transformer-based feature matching has been used to reduce errors caused by local deformation [16], and cross-modality image generation has been introduced to simplify infrared–visible registration and fusion [17]. Spatial co-registration methods have also been developed for visible, multispectral, and thermal images acquired by unmanned platforms [18]. Although these approaches are effective in certain scenarios, they often depend on feature points, dense alignment, or image-level transformation. Their performance may deteriorate in dairy barns where infrared images have weak texture, animals move, fences cause occlusion, and the robot viewpoint changes continuously. In addition, dense registration and large matching models can introduce computational burdens that are unsuitable for embedded inspection platforms. Previous studies on cattle behaviour recognition and infrared–visible registration further indicate that model complexity, weak texture, and complex scenes remain important challenges for real-time livestock monitoring [19,20,21].In addition, RGB–thermal image analysis has shown that visible and thermal information can support robust object perception when their correspondence is properly handled [22]. These findings suggest that a deployable robotic monitoring system requires not only a compact cow-head detector but also a target-level cross-camera association method that can operate in real time on edge hardware.

This study proposes YOLO11-AFE, a lightweight visible–thermal cow-head detection and heterogeneous-camera association framework for quadruped inspection robots.(Figure. 1) The framework does not perform visible–thermal image fusion or complete facial-temperature diagnosis. Instead, it focuses on two prerequisite perception tasks: detecting cow heads in both imaging modalities and establishing one-to-one target correspondence between synchronized visible-light and infrared pseudo-colour images. This target-level correspondence can provide an edge-deployable front-end basis for future local image registration, facial-region recognition, radiometric temperature mapping, and non-contact temperature extraction from regions such as the eyes, muzzle, nostrils, and ears.

The main contributions of this study are as follows. First, a lightweight cow-head detector was developed based on YOLO11n. ADown was introduced to reduce downsampling computation, SPPF_ECA was constructed by integrating efficient channel attention into SPPF, and C3k2_FE was designed to enhance adaptive feature representation in visible-light and infrared pseudo-colour images. Second, a robotic dual-camera cow-head dataset is constructed under normal illumination, weak illumination, local shadows, strong light, different viewpoints, fence occlusion, and single- and multi-cow scenes. Third, a heterogeneous-camera association strategy is proposed to handle unequal target numbers and different target positions across modalities by combining IoU and relative centre-distance cues in an improved Hungarian matching algorithm. Fourth, the complete detector–association pipeline is deployed on a Unitree Go2 quadruped robot equipped with an NVIDIA Jetson Orin NX platform, and TensorRT acceleration verifies its real-time feasibility for edge-based dairy-cow monitoring.

Figure 1. System framework of dual-modal cow-head detection and target association for the inspection robot.

Figure 1. System framework of dual-modal cow-head detection and target association for the inspection robot.

2. Materials and Methods

2.1. Data Acquisition and Processing

Data were acquired at a dairy farm of the agricultural machinery test station of the Chinese Academy of Agricultural Mechanization Sciences Group Co., Ltd., Changping District, Beijing, China. Ten barns were included. To approximate the conditions faced by an inspection robot, the collection covered different cow postures, single- and multi-animal views, fence occlusion, normal lighting, weak illumination, local shadow, and strong-light interference. The sensing system was fixed on a Unitree Go2 quadruped robot, as shown in Figure. 2, so the viewpoint was consistent with mobile inspection rather than a static camera setup.

Figure 2. Visual Perception System of the Inspection Robot.

Figure 2. Visual Perception System of the Inspection Robot.

For each acquisition cycle, visible images were captured by the left camera of a ZED2i depth camera at 1,280 x 720 pixels, and infrared pseudo-colour images were captured by a Guide Sensmart IPN384 infrared camera at 512 x 384 pixels. The SDK was extended so that both cameras were triggered at the same time. The interval between triggers was 2 s. Each trigger therefore generated one synchronized pair consisting of a visible-light image and the corresponding infrared pseudo-colour image. A total of 6,973 raw image pairs were collected.

Redundant frames were removed with the structural similarity index measure (SSIM)[23]. For adjacent frames, luminance, contrast, and structural components were compared, and 0.75 was used as the similarity threshold. This filtering discarded nearly unchanged frames while retaining changes in animal behaviour and scene configuration. After this step, 1,222 aligned synchronized image pairs were kept for dataset construction.

An asymmetric annotation scheme was adopted because the two sensors differ in resolution, field of view, and imaging mechanism[24]. LabelImg was used to annotate the two modalities separately. The visible-light class was named vis_cow_head, and the infrared class was named ir_cow_head. For visible images, the annotation rule prioritized recall: all recognizable cow heads were labelled, including targets partly covered by fences, overlapping with other animals, or truncated at image borders. For infrared pseudo-colour images, the labels were designed to support subsequent facial-temperature monitoring. Only cow heads with clear anatomical structures, visible facial thermal cues, and no serious occlusion were retained, whereas irrelevant heat sources were excluded. The final annotations included 5,683 cow-head instances in visible images and 1,951 cow-head instances in infrared pseudo-colour images.

Figure. 3 shows representative labelled samples under several lighting conditions. The visible channel changes strongly with illumination, shadow, and reflection, whereas the infrared pseudo-colour channel is more stable to ambient visible light but provides much weaker texture information.

Figure 3. Example of visible and infrared pseudo-colour images under different illumination conditions. (a) Visible-light image under normal illumination; (b) Infrared pseudo-colour image under normal illumination; (c) Visible-light image under low-illumination conditions; (d) Infrared pseudo-colour image under low-illumination conditions; (e) Visible-light image under local shadow interference; (f) Infrared pseudo-colour image under local shadow interference; (g) Visible-light image under strong-light interference; (h) Infrared pseudo-colour image under strong-light interference.

Figure 3. Example of visible and infrared pseudo-colour images under different illumination conditions. (a) Visible-light image under normal illumination; (b) Infrared pseudo-colour image under normal illumination; (c) Visible-light image under low-illumination conditions; (d) Infrared pseudo-colour image under low-illumination conditions; (e) Visible-light image under local shadow interference; (f) Infrared pseudo-colour image under local shadow interference; (g) Visible-light image under strong-light interference; (h) Infrared pseudo-colour image under strong-light interference.

The data were divided into training, validation, and test subsets at about 8:1:1, while keeping samples independent for evaluation[25]. The images covered different herd densities, from one cow to crowded multi-cow scenes. Finally, 1,956 images were used for training, 244 for validation, and 244 for testing; Table 1 gives the detailed distribution.

2.2. Construction of the Lightweight YOLO11-AFE Model

2.2.1. Improvement of the YOLO11n Model

YOLO11n was chosen as the starting detector because the task requires fast cow-head perception from visible and infrared images in complex farm scenes. Its small parameter size and high inference speed make it practical for edge devices with limited resources[26]. Nevertheless, dual-light data introduce additional difficulties: visible images are disturbed by low light, shadows, and overexposure, whereas infrared pseudo-colour images often contain weak texture and simple colour patterns. The original YOLO11n therefore needs task-specific adaptation to represent both modalities effectively.

On this basis, the YOLO11-AFE detector was designed, where A, F, and E refer to ADown, FiLM Enhancement, and Efficient Channel Attention. The network in Figure. 4 is composed of a backbone, neck, and detection head. In the backbone, some standard downsampling layers are replaced by ADown to reduce computation while retaining useful spatial responses. C3k2_FE is inserted into both the high-level backbone and the multi-scale neck to adapt features from visible and infrared inputs. ECA is further attached to SPPF, producing SPPF_ECA to strengthen multi-scale semantics and important channel responses. The Detect head from YOLO11n is retained to predict cow-head boxes at different scales. These changes aim to improve robustness to complex illumination and dual-light appearance variation while keeping the network compact.

Figure 4. Overall architecture and inference process of the YOLO11-AFE lightweight cow-head detection model. The image first enters the backbone, where multi-scale semantic features are extracted (Step 1). ADown lowers downsampling computation and retains spatial cues, C3k2_FE adjusts cow-head features for different modalities, and SPPF_ECA enlarges the receptive field while highlighting informative channels. The neck then integrates and reconstructs features at multiple scales through ADown and C3k2_FE (Step 2). The detection head predicts cow-head boxes on several feature levels (Step 3), and decoding plus NMS produces the final detections (Step 4).

Figure 4. Overall architecture and inference process of the YOLO11-AFE lightweight cow-head detection model. The image first enters the backbone, where multi-scale semantic features are extracted (Step 1). ADown lowers downsampling computation and retains spatial cues, C3k2_FE adjusts cow-head features for different modalities, and SPPF_ECA enlarges the receptive field while highlighting informative channels. The neck then integrates and reconstructs features at multiple scales through ADown and C3k2_FE (Step 2). The detection head predicts cow-head boxes on several feature levels (Step 3), and decoding plus NMS produces the final detections (Step 4).

2.2.2. Introduction of the ADown Lightweight Downsampling Module

In detection networks, downsampling decreases feature-map size and enlarges the receptive field. However, stride convolution may remove boundary details and small-target cues during compression. Cow heads in inspection images have variable scales, may be partially hidden by fences, and can be affected by strong illumination. ADown is therefore placed at selected downsampling positions. As illustrated in Figure. 5, the input feature first passes through 2 x 2 average pooling and is then split by channel. One branch performs local extraction with 3 x 3 convolution, while the other applies 3 x 3 max pooling and 1 x 1 convolution for spatial compression and channel adjustment. Concatenating the two outputs retains spatial cues from different operations and reduces the computational burden for subsequent detection.

Figure 5. Structure of the ADown lightweight downsampling module. $\mathrm{X}$ is the input feature map, ${\mathrm{X}}_1$ and ${\mathrm{X}}_2$ are the two branch features obtained by channel-wise split, ${\mathrm{Y}}_1$ is the output of the upper branch convolution, ${\mathrm{Y}}_2$ is the output of the lower branch pooling and convolution, Y is the output feature of the module; C is the number of channels in the feature map, H and W are the height and width of the feature map, s denotes the stride of pooling, and p denotes the padding of convolution.

Figure 5. Structure of the ADown lightweight downsampling module. $\mathrm{X}$ is the input feature map, ${\mathrm{X}}_1$ and ${\mathrm{X}}_2$ are the two branch features obtained by channel-wise split, ${\mathrm{Y}}_1$ is the output of the upper branch convolution, ${\mathrm{Y}}_2$ is the output of the lower branch pooling and convolution, Y is the output feature of the module; C is the number of channels in the feature map, H and W are the height and width of the feature map, s denotes the stride of pooling, and p denotes the padding of convolution.

2.2.3. Construction of the C3k2_FE Feature-Adaptive Enhancement Module

The two imaging modalities provide complementary but inconsistent information. Visible images contain rich edges and local textures, but their quality changes with illumination. Infrared pseudo-colour images encode thermal radiation and are less affected by visible light, yet they usually lack fine texture. Such modality differences can reduce the generalisation of a detector trained on mixed dual-light data. To address this issue, the C3k2_FE module, C3k2 with FiLM Enhancement, is inserted into the high-level feature-extraction part of the backbone and the multi-scale reconstruction part of the neck, as shown in Figure. 6.

Figure 6. Structure of the FiLM-Enhanced C3k2_FE Module. $\mathrm{X}$ represents the input feature of the C3k2_FE module, and $\mathrm{Y}$ represents the output feature. FE denotes the feature-adaptive enhancement module, FiLM denotes feature-wise linear modulation, and High-frequency denotes the high-frequency residual enhancement branch.

Figure 6. Structure of the FiLM-Enhanced C3k2_FE Module. $\mathrm{X}$ represents the input feature of the C3k2_FE module, and $\mathrm{Y}$ represents the output feature. FE denotes the feature-adaptive enhancement module, FiLM denotes feature-wise linear modulation, and High-frequency denotes the high-frequency residual enhancement branch.

C3k2_FE augments the original C3k2 block with a FiLM branch (Figure. 7). A global descriptor is first obtained from the input feature by global average pooling. This descriptor is then fed into three lightweight branches: an Alpha branch that predicts interpolation weights and two FiLM branches that generate modality-related affine parameters. Each FiLM branch uses two consecutive 1 x 1 convolutions with SiLU activation. The first convolution compresses the channel dimension for inexpensive transformation, and the second outputs twice the input-channel number, which is separated into scale and shift terms. The Alpha branch uses 1 x 1 convolutions, SiLU, and Sigmoid to constrain the interpolation weights to [0, 1]. The two sets of affine parameters are blended with these weights and applied to the input feature along the channel dimension. Because the modulation is inferred from global feature statistics rather than explicit modality labels, the block can adjust channel responses adaptively and reduce fluctuations caused by cross-modal differences.

$$\mathrm{G}=\mathrm{GlobalAvgPool}2\mathrm{d}\left(\mathrm{X}\right),\mathrm{G}\in {\mathbb{R}}^{\mathrm{C}\text{×}1\text{×}1}$$

(1)

$$\left({\mathsf{\gamma }}_{\mathrm{vis}},{\mathsf{\beta }}_{\mathrm{vis}}\right)={\mathrm{Conv}}_{\mathrm{vis}}\left(\mathrm{G}\right),\left({\mathsf{\gamma }}_{\mathrm{ir}},{\mathsf{\beta }}_{\mathrm{ir}}\right)={\mathrm{Conv}}_{\mathrm{ir}}\left(\mathrm{G}\right)$$

(2a,2b)

$$\mathsf{\gamma }=\mathrm{w}·{\mathsf{\gamma }}_{\mathrm{vis}}+(1-\mathrm{w})·{\mathsf{\gamma }}_{\mathrm{ir}},\mathsf{\beta }=\mathrm{w}·{\mathsf{\beta }}_{\mathrm{vis}}+(1-\mathrm{w})·{\mathsf{\beta }}_{\mathrm{ir}}$$

(3a,3b)

$${{\mathrm{X}}_1}^{\text{'}}=\mathsf{\gamma }\odot \mathrm{X}+\mathsf{\beta }$$

(4)

where G∈ℝC×1×1 represents the channel-wise context vector after global pooling, γvis,βvis,γir,βir denote the affine parameters generated by the visible-light and infrared branches,w is the learnable interpolation coefficient,${{\mathrm{X}}_1}^{\text{'}}$ is the channel-modulated output feature, and ⊙ represents channel-wise element-wise multiplication.

Figure 7. Structure of the FiLM branch in the C3k2_FE module. ${\mathrm{X}}_1$ is the input feature; GAP denotes global average pooling; Alpha, vis head, and ir head represent the branches generating the learnable interpolation weight, visible-light affine parameters, and infrared affine parameters, respectively; $\mathrm{w}$ is the learnable interpolation weight; ${{\mathrm{X}}_1}^{\text{'}}$ is the output feature after channel-wise FiLM modulation.

Figure 7. Structure of the FiLM branch in the C3k2_FE module. ${\mathrm{X}}_1$ is the input feature; GAP denotes global average pooling; Alpha, vis head, and ir head represent the branches generating the learnable interpolation weight, visible-light affine parameters, and infrared affine parameters, respectively; $\mathrm{w}$ is the learnable interpolation weight; ${{\mathrm{X}}_1}^{\text{'}}$ is the output feature after channel-wise FiLM modulation.

A high-frequency residual branch is also added to emphasize detailed cow-head structures (Figure. 8). This branch obtains a low-frequency component through local 3 x 3 average pooling and subtracts it from the FiLM-modulated feature to isolate high-frequency information. The residual is refined with 3 x 3 depthwise convolution and batch normalisation, and is then added back to the FiLM-modulated feature while preserving the output-channel number. A learnable scaling factor controls the contribution of the residual and is optimized during training through the detection loss. In this way, the model can selectively strengthen edges, contours, and local texture cues in scenes with harsh illumination, weak infrared texture, or occlusion.

$${\mathrm{X}}_{\mathrm{low}}={\mathrm{AvgPool}}_{3\text{×}3,\mathrm{s}=1,\mathrm{p}=1}\left({{\mathrm{X}}_1}^{\text{'}}\right)$$

(5)

$${\mathrm{X}}_{\mathrm{hf}}={{\mathrm{X}}_1}^{\text{'}}-{\mathrm{X}}_{\mathrm{low}}$$

(6)

$${\widehat{\mathrm{X}}}_{\mathrm{hf}}=\mathrm{BN}\left({\mathrm{DWConv}}_{3\text{×}3}\left({\mathrm{X}}_{\mathrm{hf}}\right)\right)$$

(7)

$$\mathrm{Y}={{\mathrm{X}}_1}^{\text{'}}+\mathsf{\tau }·{\widehat{\mathrm{X}}}_{\mathrm{hf}}$$

(8)

where ${{\mathrm{X}}_1}^{\text{'}}$denotes the feature after FiLM modulation; ${\mathrm{X}}_{\mathrm{low}}$ denotes the low-frequency component extracted by 3×3 average pooling; ${\mathrm{X}}_{\mathrm{hf}}$ denotes the high-frequency residual; ${\mathrm{DWConv}}_{3\text{×}3}$ denotes the 3×3 depthwise convolution; BN($\bullet$) denotes the batch normalization operation;${\widehat{\mathrm{X}}}_{\mathrm{hf}}$ denotes the high-frequency feature enhanced by depthwise convolution and batch normalization;$\mathsf{\tau }$ denotes the learnable scaling coefficient; and Y denotes the output feature obtained by fusing the high-frequency residual enhancement branch with the FiLM-modulated feature.

Figure 8. Structure of the high-frequency residual enhancement branch. ${{\mathrm{X}}_1}^{\text{'}}$is the input feature after FiLM modulation; Y is the output feature after high-frequency residual enhancement.

Figure 8. Structure of the high-frequency residual enhancement branch. ${{\mathrm{X}}_1}^{\text{'}}$is the input feature after FiLM modulation; Y is the output feature after high-frequency residual enhancement.

2.2.4. SPPF_ECA Multi-Scale Channel Attention Enhancement Module

SPPF obtains multi-scale semantics by repeated max pooling and increases the receptive field with little extra computation. Its operation, however, mainly fuses information in the spatial domain and does not explicitly reweight channels. In dual-light cow-head detection, visible and infrared inputs activate different informative channels, so spatial pyramid pooling alone may not sufficiently emphasize cow-head-related responses. Therefore, ECA is appended after SPPF to produce the SPPF_ECA module shown in Figure. 9.

In ECA, each channel is first compressed to a scalar by adaptive global average pooling, yielding a compact descriptor:

$${\mathrm{Z}}_{\mathrm{c}}=\mathrm{AdaptiveAvgPool}2\mathrm{d}\left({\mathrm{F}}_{\mathrm{c}}\right),\text{∀}\mathrm{c}\text{∈}\text{{}1,2,\text{…},\mathrm{C}\text{}}$$

(9)

where ${\mathrm{F}}_{\mathrm{c}}$ denotes the feature map of the $\mathrm{c}$ channel.

Neighbouring-channel interactions are then modelled with a one-dimensional convolution whose kernel size is chosen from the channel number. Sigmoid activation converts the response into channel weights:

$${\mathrm{K}}_{\mathrm{ec}}={\left|{\displaystyle \frac{{\log }_2(\mathrm{C})+\mathrm{b}}{\mathsf{\gamma }}}\right|}_{\mathrm{odd}}$$

(10)

$$\mathrm{s}=\mathsf{\sigma }\left({\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}1\mathrm{D}}_{{\mathrm{k}}_{\mathrm{e}\mathrm{c}}}\left(\mathrm{z}\right)\right)$$

(11)

where and are the tuning factors, commonly set to 1 and 2[27]. The rounding operation ensures an odd kernel size for centre alignment, the Sigmoid function normalizes the response, and the resulting vector stores channel-wise weights.

The feature map is multiplied by these weights to obtain the recalibrated output:

$${\mathrm{F}}^{\text{‘}}=\mathrm{F}\text{⨀}\mathrm{s}$$

(12)

where $\text{⨀}$ denotes channel-wise element-wise multiplication.

Unlike channel-attention designs that compress channels with fully connected layers, ECA keeps the channel dimension unchanged and adds little computation. It therefore improves important channel responses without compromising lightweight inference. In the proposed network, SPPF_ECA enhances high-level cow-head semantics and improves discrimination from complex backgrounds.

Figure 9. Structure of the SPPF_ECA multi-scale channel attention module. $\mathrm{X}$ denotes the input feature of the SPPF_ECA module; F₀、F₁、F₂、F₃ denote the multi-scale features obtained by 1×1 convolution and successive max-pooling operations; F denotes the SPPF output feature after feature concatenation and 1×1 convolution fusion; Y denotes the output feature recalibrated by ECA channel attention.

Figure 9. Structure of the SPPF_ECA multi-scale channel attention module. $\mathrm{X}$ denotes the input feature of the SPPF_ECA module; F₀、F₁、F₂、F₃ denote the multi-scale features obtained by 1×1 convolution and successive max-pooling operations; F denotes the SPPF output feature after feature concatenation and 1×1 convolution fusion; Y denotes the output feature recalibrated by ECA channel attention.

Figure 10. Structure of the Efficient Channel Attention (ECA) module. GAP denotes global average pooling;Kec denotes the kernel size of the 1D convolution in ECA; $\mathsf{\sigma }$ denotes the Sigmoid activation function; $\odot$ denotes element-wise multiplication.

Figure 10. Structure of the Efficient Channel Attention (ECA) module. GAP denotes global average pooling;Kec denotes the kernel size of the 1D convolution in ECA; $\mathsf{\sigma }$ denotes the Sigmoid activation function; $\odot$ denotes element-wise multiplication.

2.3. Improvement of the Heterogeneous-Camera Target Association Algorithm

In a moving robot system, the visible and infrared cameras are installed at different positions and differ in resolution and field of view. The same cow head may therefore appear at different scales, positions, and visual forms in the two modalities. Animal movement, occlusion by fences, pose changes, and missed detections can also make the target numbers inconsistent between synchronized image pairs. Pixel-level matching is unsuitable in this setting because it is sensitive to the appearance gap between texture images and thermal pseudo-colour images and because it introduces a large computational burden for edge processing.

The association module is built on the detection boxes produced by YOLO11-AFE. Visible-light boxes and infrared pseudo-colour boxes are first mapped into a normalized coordinate space, after which target-level matching is used to determine whether two boxes belong to the same cow head. Unlike dense registration or deep re-identification, the cost computation uses only box locations and overlap relations. This choice keeps the algorithm simple and makes it appropriate for deployment on embedded inspection platforms.

2.3.1. Problem Definition and Overall Workflow

YOLO11-AFE outputs two bounding-box sets from synchronized visible-light and infrared pseudo-colour frames. They are defined as follows:

$${\mathrm{B}}^{\mathrm{vis}}=\left\{{\mathrm{b}}_1^{\mathrm{vis}},{\mathrm{b}}_2^{\mathrm{vis}},\text{…},{\mathrm{b}}_{\mathrm{m}}^{\mathrm{vis}}\right\}，{\mathrm{B}}^{\mathrm{ir}}=\left\{{\mathrm{b}}_1^{\mathrm{ir}},{\mathrm{b}}_2^{\mathrm{ir}},\text{…},{\mathrm{b}}_{\mathrm{n}}^{\mathrm{ir}}\right\}$$

(13a,13b)

where ${\mathrm{B}}^{\mathrm{vis}}$ and ${\mathrm{B}}^{\mathrm{ir}}$ denote the sets of cow-head bounding boxes in the visible-light and infrared pseudo-colour images, respectively; (m) and (n) denote the numbers of cow-head targets detected in the visible-light and infrared pseudo-colour images, respectively; and ${\mathrm{b}}_{\mathrm{m}}^{\mathrm{vis}}$ and ${\mathrm{b}}_{\mathrm{n}}^{\mathrm{ir}}$ denote the i-th visible-light cow-head bounding box and the j-th infrared pseudo-colour cow-head bounding box, respectively.

Because occlusion, missed detection, and camera field-of-view differences can make the two target counts unequal, the goal is not to pair every detection blindly. Instead, the task is to obtain the most plausible correspondence between the visible and infrared box sets:

$$\mathrm{M}=\left\{\left({\mathrm{b}}_{\mathrm{i}}^{\mathrm{vis}},{\mathrm{b}}_{\mathrm{j}}^{\mathrm{ir}}\right)\text{∣}\mathrm{i}\text{∈}[1,\mathrm{m}],\mathrm{j}\text{∈}[1,\mathrm{n}]\right\}$$

(14)

where M denotes the set of matched cow-head targets between the visible-light and infrared pseudo-colour images.

If a pair is accepted, the two boxes are treated as observations of the same cow head; otherwise, the candidate is discarded as unmatched. The proposed workflow first builds a cost matrix from two spatial cues: box overlap and the relative distance of each box centre to the corresponding image centre. The Hungarian algorithm then searches for the minimum-cost assignment, and additional thresholds remove abnormal pairs. This procedure yields one-to-one correspondence for multi-cow dual-light scenes.

2.3.2. Construction of the Fused Cost Matrix

A cow head captured by the two cameras usually preserves approximate relative spatial consistency, but its absolute pixel coordinates differ because the sensors have different resolutions, views, and mounting positions. Directly measuring the pixel distance between two box centres can therefore be misleading. To reduce sensitivity to these factors, each box centre is compared with the centre of its own image, and the difference between the two centre-distance values is used as a spatial consistency term. The IoU of the two boxes is then incorporated to represent regional overlap, and the two measurements jointly form the fused cost matrix.

For a visible-light candidate and an infrared candidate, their box-centre coordinates are written as:

$${\mathrm{p}}_{\mathrm{i}}^{\mathrm{vis}}=\left({\mathrm{x}}_{\mathrm{i}}^{\mathrm{vis}},{\mathrm{y}}_{\mathrm{i}}^{\mathrm{vis}}\right),{\mathrm{p}}_{\mathrm{j}}^{\mathrm{ir}}=\left({\mathrm{x}}_{\mathrm{j}}^{\mathrm{ir}},{\mathrm{y}}_{\mathrm{j}}^{\mathrm{ir}}\right)$$

(15a,15b)

The centres of the corresponding visible-light and infrared pseudo-colour images are defined as:

$${\mathrm{o}}^{\mathrm{vis}}=\left({\mathrm{x}}_0^{\mathrm{vis}},{\mathrm{y}}_0^{\mathrm{vis}}\right),{\mathrm{o}}^{\mathrm{ir}}=\left({\mathrm{x}}_0^{\mathrm{ir}},{\mathrm{y}}_0^{\mathrm{ir}}\right)$$

(16a,16b)

The distance from the visible-light box centre to the visible-light image centre is calculated by:

$${\mathrm{d}}_{\mathrm{i}}^{\mathrm{vis}}=\sqrt{{\left({\mathrm{x}}_{\mathrm{i}}^{\mathrm{vis}}-{\mathrm{x}}_0^{\mathrm{vis}}\right)}^2+{\left({\mathrm{y}}_{\mathrm{i}}^{\mathrm{vis}}-{\mathrm{y}}_0^{\mathrm{vis}}\right)}^2}$$

(17)

The same calculation is performed for the infrared pseudo-colour box centre and its image centre:

$${\mathrm{d}}_{\mathrm{j}}^{\mathrm{ir}}=\sqrt{{\left({\mathrm{x}}_{\mathrm{j}}^{\mathrm{ir}}-{\mathrm{x}}_0^{\mathrm{ir}}\right)}^2+{\left({\mathrm{y}}_{\mathrm{j}}^{\mathrm{ir}}-{\mathrm{y}}_0^{\mathrm{ir}}\right)}^2}$$

(18)

The relative location discrepancy of the two candidates is then expressed by the normalized difference between these two distances:

$${\mathrm{D}}_{\mathrm{ij}}={\displaystyle \frac{\left|{\mathrm{d}}_{\mathrm{i}}^{\mathrm{vis}}-{\mathrm{d}}_{\mathrm{j}}^{\mathrm{ir}}\right|}{{\mathrm{d}}_{\mathrm{m}\mathrm{a}\mathrm{x}}}}$$

(19)

where ${\mathrm{p}}_{\mathrm{i}}^{\mathrm{vis}}$ and ${\mathrm{p}}_{\mathrm{j}}^{\mathrm{ir}}$ denote the centre coordinates of the visible-light and infrared pseudo-colour detection boxes, respectively; ${\mathrm{o}}^{\mathrm{vis}}$ and ${\mathrm{o}}^{\mathrm{ir}}$ denote the centre coordinates of the visible-light and infrared pseudo-colour images, respectively; ${\mathrm{d}}_{\mathrm{i}}^{\mathrm{vis}}$ and ${\mathrm{d}}_{\mathrm{j}}^{\mathrm{ir}}$ denote the Euclidean distances from the detection-box centres to the corresponding image centres; ${\mathrm{D}}_{\mathrm{ij}}$ denotes the normalized relative centre-distance difference; and ${\mathrm{d}}_{\mathrm{m}\mathrm{a}\mathrm{x}}$is the normalization scale factor, which can be set as the diagonal length of the corresponding image or the maximum possible distance difference. A smaller ${\mathrm{D}}_{\mathrm{ij}}$ indicates that the two targets have more similar relative spatial positions in their respective images.

IoU is used to measure how much the two candidate boxes overlap:

$$\mathrm{Io}{\mathrm{U}}_{\mathrm{ij}}={\displaystyle \frac{{\mathrm{b}}_{\mathrm{i}}^{\mathrm{vis}}\text{∩}{\mathrm{b}}_{\mathrm{j}}^{\mathrm{ir}}}{{\mathrm{b}}_{\mathrm{i}}^{\mathrm{vis}}\text{∪}{\mathrm{b}}_{\mathrm{j}}^{\mathrm{ir}}}}$$

(20)

where $\mathrm{Io}{\mathrm{U}}_{\mathrm{ij}}$ denotes the intersection over union between the i-th visible-light detection box and the j-th infrared pseudo-colour detection box; ${\mathrm{b}}_{\mathrm{i}}^{\mathrm{vis}}\text{∩}{\mathrm{b}}_{\mathrm{j}}^{\mathrm{ir}}$ denotes the intersection area of the two detection boxes; and ${\mathrm{b}}_{\mathrm{i}}^{\mathrm{vis}}\text{∪}{\mathrm{b}}_{\mathrm{j}}^{\mathrm{ir}}$ denotes their union area. A larger $\mathrm{Io}{\mathrm{U}}_{\mathrm{ij}}$ indicates a higher degree of spatial overlap between the two detection boxes.

The matching cost for a visible-light target and an infrared pseudo-colour target is then defined by combining the normalized centre-distance difference and the IoU term:

$${\mathrm{C}}_{\mathrm{ij}}=\mathsf{\lambda }{\mathrm{D}}_{\mathrm{ij}}+(1-\mathsf{\lambda })\left(1-\mathrm{Io}{\mathrm{U}}_{\mathrm{ij}}\right)$$

(21)

where ${\mathrm{C}}_{\mathrm{ij}}$ denotes the matching cost of the target pair (${\mathrm{b}}_{\mathrm{i}}^{\mathrm{vis}},{\mathrm{b}}_{\mathrm{j}}^{\mathrm{ir}}$); $\mathsf{\lambda }$ is the weighting coefficient used to balance the effects of the relative centre-distance difference and IoU on the matching cost; and $1-\mathrm{Io}{\mathrm{U}}_{\mathrm{ij}}$ is the overlap penalty term. A smaller ${\mathrm{C}}_{\mathrm{ij}}$ indicates a higher probability that the visible-light target and the infrared pseudo-colour target correspond to the same cow head.

After the pairwise costs are obtained, they are arranged into the heterogeneous-camera association matrix:

$$\mathrm{C}=\left[\begin{array}{cccc}{\mathrm{C}}_{11}& {\mathrm{C}}_{12}& \text{…}& {\mathrm{C}}_{1\mathrm{n}}\\ {\mathrm{C}}_{21}& {\mathrm{C}}_{22}& \text{…}& {\mathrm{C}}_{2\mathrm{n}}\\ \text{⋮}& \text{⋮}& \text{⋱}& \text{⋮}\\ {\mathrm{C}}_{\mathrm{m}1}& {\mathrm{C}}_{\mathrm{m}2}& \text{…}& {\mathrm{C}}_{\mathrm{mn}}\end{array}\right]$$

(22)

where $\mathrm{C}$ denotes the heterogeneous-camera target association cost matrix with a dimension of m×n, and each element represents the matching cost between one visible-light target and one infrared pseudo-colour target.

2.3.3. Improved Hungarian Matching and Outlier Association Removal

Given the fused cost matrix, the Hungarian algorithm is adopted to find the minimum-cost one-to-one assignment between visible-light and infrared pseudo-colour detections. This global optimisation avoids the duplicated or incorrect matches that may occur with local nearest-neighbour rules[28]. The objective is:

$${\mathrm{M}}^{\mathrm{*}}=\arg \underset{\mathrm{M}}{\min }\sum _{(\mathrm{i},\mathrm{j})\text{∈}\mathrm{M}}{\mathrm{C}}_{\mathrm{ij}}$$

(23)

where ${\mathrm{M}}^{\mathrm{*}}$ denotes the set of optimal matches, $\mathrm{M}$ denotes all feasible match combinations, and ${\mathrm{C}}_{\mathrm{ij}}$ represents the association cost between the i-th visible-light target and the j-th infrared pseudo-colour target.

In real inspection data, some detections should not be assigned because the two cameras may capture different target sets. If every Hungarian output were retained, boxes with large relative displacement or negligible overlap could be incorrectly forced into pairs. For this reason, threshold filtering is applied after assignment. A candidate match is kept only when:

$${\mathrm{C}}_{\mathrm{ij}}\text{<}{\mathrm{T}}_{\mathrm{c}}$$

(24)

with additional constraints on relative centre-distance difference and IoU:

$${\mathrm{D}}_{\mathrm{ij}}\text{<}{\mathrm{T}}_{\mathrm{d}},\mathrm{Io}{\mathrm{U}}_{\mathrm{ij}}\text{>}{\mathrm{T}}_{\mathrm{iou}}$$

(25)

where ${\mathrm{T}}_{\mathrm{c}}$ is the cost threshold, ${\mathrm{T}}_{\mathrm{d}}$ is the relative centre-distance threshold, and ${\mathrm{T}}_{\mathrm{iou}}$ is the IoU threshold.

Candidates violating any threshold are discarded, giving the valid matching set:

$${\mathrm{M}}_{\mathrm{valid}}=\text{{}({\mathrm{b}}_{\mathrm{i}}^{\mathrm{vis}},{\mathrm{b}}_{\mathrm{j}}^{\mathrm{ir}})\text{∣}{\mathrm{C}}_{\mathrm{ij}}\text{<}{\mathrm{T}}_{\mathrm{c}},{\mathrm{D}}_{\mathrm{ij}}\text{<}{\mathrm{T}}_{\mathrm{d}},\mathrm{Io}{\mathrm{U}}_{\mathrm{ij}}\text{>}{\mathrm{T}}_{\mathrm{iou}}\text{}}$$

(26)

where ${\mathrm{M}}_{\mathrm{valid}}$ denotes the set of matches retained after outlier removal.

In summary, the association method uses YOLO11-AFE detections as its input and constructs a lightweight cost matrix from centre-distance discrepancy and IoU. The improved Hungarian matching then provides one-to-one correspondence between visible-light and infrared pseudo-colour cow-head targets. Because no pixel-level cross-modal registration is required, the method remains efficient and can handle occlusion, target overlap, and unequal target counts while supplying target-level correspondence for future non-contact facial-temperature monitoring in dairy cows.

2.4. Hardware Environment and Evaluation Metrics

The experiments were run on a server equipped with an NVIDIA RTX A6000 GPU. The software stack consisted of Ubuntu 20.04.6 LTS, Python 3.10.19, PyTorch 2.9.1, and CUDA 12.8.

For training, images were resized to 640 x 640 pixels, the batch size was 64, and 200 epochs were scheduled with 8 data-loading workers. Stochastic gradient descent was used for optimisation, and the random seed was fixed at 523. To stabilize training and reduce overfitting, early stopping was enabled with patience set to 50.

Model accuracy was reported with precision (P), mAP at an IoU threshold of 0.5, and mAP averaged over IoU thresholds from 0.5 to 0.95. Computational complexity was described using the number of trainable parameters and GFLOPs. The following equations define these metrics:

$$\mathrm{P}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(27)

$$\mathrm{AP}={\int }_0^1\mathrm{P}\left(\mathrm{R}\right)\mathrm{dR}$$

(28)

$$\mathrm{mAP}={\displaystyle \frac{1}{\mathrm{N}}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}}\mathrm{A}{\mathrm{P}}_{\mathrm{i}}$$

(29)

where $\mathrm{TP}$ denotes the number of true positive detections, $\mathrm{FP}$ denotes the number of false positive detections, $\mathrm{P}\left(\mathrm{R}\right)$ represents the precision–recall curve, $\mathrm{AP}$ denotes the average precision for a single class, $\mathrm{A}{\mathrm{P}}_{\mathrm{i}}$ denotes the average precision of the i-th class, and $\mathrm{N}$ is the total number of target classes.

3. Results

3.1. Comparison of Detection Models

Cow-head detection was evaluated separately for the visible-light and infrared pseudo-colour test subsets. YOLO11-AFE was benchmarked against RT-DETR_L, Faster-RCNN, YOLOv5n, YOLOv8n, YOLOv10n, YOLO11n, and YOLOv12n. To ensure an unbiased comparison, all models were trained again with the same data split, input resolution, and training configuration before being tested on both modalities. The results are given in Table 2.

As Table 2 indicates, YOLO11-AFE obtained the strongest overall performance for the two modalities. On the visible-light subset, Precision, mAP@0.5, and mAP@0.5:0.95 were 97.51%, 96.26%, and 70.64%, respectively, higher than those of the comparison models. This suggests improved tolerance to illumination changes, cluttered backgrounds, and multi-cow scenes. On the infrared pseudo-colour subset, the corresponding values reached 95.92%, 99.07%, and 91.41%, showing stable detection even when thermal texture is weak and object boundaries are unclear.

Relative to YOLO11n, the proposed network increased accuracy while remaining compact. For visible-light images, the gains in Precision, mAP@0.5, and mAP@0.5:0.95 were 4.40, 2.15, and 1.36 percentage points. For infrared pseudo-colour images, the gains were 1.65, 0.15, and 1.15 percentage points. The model size decreased from 2.59M to 2.14M parameters, and computation decreased from 6.44 to 5.27 GFLOPs, corresponding to reductions of 17.37% and 18.17%. These results show that ADown, C3k2_FE, and SPPF_ECA improve cross-modal cow-head representation while reducing complexity. Among all compared networks, YOLO11-AFE provided the smallest parameter and FLOP values together with the highest accuracy, indicating a favourable balance for edge deployment.

Figure 11. Qualitative comparison of cow-head detection results between YOLO11n and YOLO11-AFE under different complex dairy-farm scenes. In each panel, the upper images show the results of YOLO11n, and the lower images show the results of YOLO11-AFE. (a) Strong-light interference scene; (b) normal illumination scene; (c) local shadow interference scene; (d) low-illumination scene.

Figure 11. Qualitative comparison of cow-head detection results between YOLO11n and YOLO11-AFE under different complex dairy-farm scenes. In each panel, the upper images show the results of YOLO11n, and the lower images show the results of YOLO11-AFE. (a) Strong-light interference scene; (b) normal illumination scene; (c) local shadow interference scene; (d) low-illumination scene.

Four difficult farm conditions were selected for qualitative comparison with YOLO11n: strong-light interference, dense multi-cow distribution, local shadow, and low illumination. Figure. 11 shows that YOLO11-AFE gives more stable cow-head locations and higher confidence scores. The improvement is consistent with the roles of the three modules: ADown alleviates information loss in downsampling, C3k2_FE improves adaptation between visible and infrared appearances and enhances fine details, and SPPF_ECA reweights important high-level channels. These visual results support the quantitative finding that the model is more robust across challenging dual-light farm scenes.

3.2. Ablation Study Analysis

The influence of ADown, C3k2_FE, and SPPF_ECA was analysed through ablation experiments, with YOLO11n serving as the reference model. Table 3 lists the results. When only ADown was introduced, parameters decreased from 2.59M to 2.04M and FLOPs from 6.44 to 5.17, reductions of 21.24% and 19.72%. At the same time, visible-light Precision rose from 93.11% to 97.37%, and infrared mAP@0.5:0.95 rose from 90.26% to 91.01%, indicating that ADown preserves useful features while lowering computation. Adding only C3k2_FE raised visible-light mAP@0.5 and mAP@0.5:0.95 by 1.75 and 1.62 percentage points and raised infrared mAP@0.5:0.95 by 1.19 percentage points, showing stronger dual-modal representation. When only SPPF_ECA was used, visible-light Precision and mAP@0.5 increased by 3.15 and 1.55 percentage points, confirming the benefit of channel reweighting in high-level features.

The combined settings further reveal that the modules are complementary. ADown plus C3k2_FE kept parameters and FLOPs at 2.14M and 5.27 while maintaining good accuracy for both modalities. C3k2_FE together with SPPF_ECA raised visible-light Precision to 97.43%, implying that modality adaptation and channel attention reinforce each other. The complete YOLO11-AFE model used 2.14M parameters and 5.27 GFLOPs, which are 17.37% and 18.17% lower than the baseline. It achieved 97.51%, 96.26%, and 70.64% for visible-light Precision, mAP@0.5, and mAP@0.5:0.95, with gains of 4.40, 2.15, and 1.36 percentage points. For infrared pseudo-colour images, the metrics reached 95.92%, 99.07%, and 91.41%, with gains of 1.65, 0.15, and 1.15 percentage points. These ablations confirm that the three components jointly improve accuracy and efficiency, making the detector suitable for inspection-robot edge devices.

3.3. Heterogeneous-Camera Target Association Results Analysis

The synchronized dual-light test set was used to evaluate the heterogeneous-camera association module. YOLO11-AFE first produced cow-head boxes in the visible-light and infrared pseudo-colour images. These boxes were then matched by the improved Hungarian algorithm, whose cost integrates IoU and Euclidean distance between target-box centres. The overall association results are summarized in Table 4.

The target association accuracy is computed as:

$$\mathrm{Ac}{\mathrm{c}}_{\mathrm{match}}={\displaystyle \frac{{\mathrm{N}}_{\mathrm{correct}}}{{\mathrm{N}}_{\mathrm{total}}}}\text{×}100\%$$

(30)

where $\mathrm{Ac}{\mathrm{c}}_{\mathrm{match}}$ denotes the target association accuracy, ${\mathrm{N}}_{\mathrm{correct}}$ represents the number of correctly matched cow-head target pairs, and ${\mathrm{N}}_{\mathrm{total}}$ is the total number of cow-head target pairs determined from manual annotation.

Table 4 reports 187 manually confirmed cow-head pairs from 122 synchronized dual-light test-image pairs. The algorithm associated all 187 pairs correctly, and neither mismatches nor missed matches occurred; the matching accuracy was therefore 100%. This result indicates that reliable correspondence can be established between visible-light and infrared pseudo-colour targets. The test set contains multi-cow scenes, partial occlusions, and inconsistent target numbers, where direct pairing can easily fail. By using IoU and relative centre-distance difference in a global assignment framework, the method satisfies the one-to-one association requirement for dual-light cow-head targets.

Figure. 12 shows examples of heterogeneous-camera association. Lines connect boxes that correspond to the same cow across the visible-light and infrared pseudo-colour views. The examples demonstrate that the algorithm can maintain cross-modal matching in multi-cow scenes and under spatial offset. Matches that do not pass the constraints are removed from the final set, which avoids erroneous forced associations. This provides a dependable basis for subsequent local registration, facial-region recognition, and radiometric temperature extraction at the same inspection moment.

Figure 12. Visualization results of cow-head target association between heterogeneous cameras. (a) Association result in a single-target scene; (b) Association result in a two-target scene; (c) Association result in a three-target scene; (d) Association result in a four-target scene.

Figure 12. Visualization results of cow-head target association between heterogeneous cameras. (a) Association result in a single-target scene; (b) Association result in a two-target scene; (c) Association result in a three-target scene; (d) Association result in a four-target scene.

3.4. Edge-Device Deployment Performance Analysis

The proposed system was deployed on a quadruped inspection robot to evaluate real-time edge performance. The robot carried an NVIDIA Jetson Orin NX 16GB computing module, and the runtime environment included Jetpack 5.1.1, CUDA 11.4, cuDNN 8.6.0, and TensorRT 8.6.0. The platform provides about 100 TOPS of AI computing capability with a maximum power consumption of 25 W. Native PyTorch and TensorRT-accelerated inference were both tested. Figure. 13 presents the deployment interface, and Table 5 summarizes average inference time, end-to-end processing time, and frame rate.

Figure 13. Deployment results on the inspection robot edge computing platform.

Figure 13. Deployment results on the inspection robot edge computing platform.

Under PyTorch, the average inference time was 38.14 ms, giving 26.22 FPS. Including inference, bounding-box parsing, and target association, the core process required 40.81 ms, equivalent to 24.50 FPS. With TensorRT, inference time fell to 24.23 ms and the inference speed increased to 41.26 FPS; the core-process time was shortened to 27.24 ms, or 36.71 FPS. Relative to PyTorch, TensorRT reduced inference time by 36.47%, raised inference FPS by 57.36%, reduced core-process time by 33.25%, and increased core-process FPS by 49.84%.

The Hungarian matching step required only 0.12 ms under both inference modes. Its proportions of the core-process time were 0.289% for PyTorch and 0.444% for TensorRT, showing that association is not the runtime bottleneck. These deployment results show that the pipeline can operate in real time on the inspection-robot edge platform, while TensorRT provides further acceleration. Thus, the system meets the practical requirements of cow-head detection and heterogeneous-camera association in complex farm environments.

4. Discussion

This study developed a compact cow-head detector and a heterogeneous-camera association method for robotic barn inspection. A visible–thermal acquisition and analysis platform was constructed on a quadruped robot with edge computing. In complex dairy-farm scenes, the platform achieved real-time cow-head detection and cross-camera association, providing target-level correspondence between visible-light and infrared pseudo-colour images. Recent studies have shown that facial thermal regions in cattle, including the eyes, muzzle, nostrils, ears, and horns, can be automatically located or analysed for non-contact temperature monitoring [29,30]. Therefore, the present framework can be regarded as a front-end perception module for future facial-temperature extraction in robotic dairy monitoring.

4.1. Innovation

Many existing cattle-perception studies use fixed cameras or a single visible-light stream for detection, tracking, or identity recognition. These works are useful for farm perception, but a mobile inspection robot must deal with additional factors, including crowded cows, fence occlusion, viewpoint variation, overexposure, weak illumination, and low-texture infrared images. Compared with those studies, the present work moves from single-modal cow detection to visible–thermal cow-head detection and target-level cross-camera association on a moving robot. The obtained correspondence between visible-light and infrared pseudo-colour images provides the first step toward later local registration, facial-region recognition, and radiometric temperature extraction.

The comparative experiments were carried out with representative detection models under the same conditions. All models used the same visible–thermal cow-head dataset, data split, input resolution, training epochs, and hardware platform. The results show that YOLO11-AFE maintained high accuracy and stable detection performance under complex illumination, weak infrared texture, and multi-cow scenes, while reducing parameters and computational cost. This balance between accuracy, robustness, and lightweight computation is important for practical deployment on inspection-robot edge hardware.

The target-association design is also different from image-level cross-modal registration. Instead of aligning whole visible-light and infrared images, it uses detected cow-head boxes as the association units. The cost matrix integrates IoU and centre-distance difference relative to the image centre, and the improved Hungarian matching algorithm selects the global assignment while removing abnormal pairs. Owing to this target-level formulation, the method has low computational cost and is suitable for real-time processing on embedded robot hardware.

4.2. Application Scenario

The proposed method can serve as the front-end perception module for non-contact facial-temperature monitoring during robotic dairy inspection. It detects cow heads in synchronized visible-light and infrared pseudo-colour images and associates detections belonging to the same animal across the two camera views. This target-level correspondence is important in multi-cow scenes, because thermal information must be assigned to the correct individual before it can support animal-level monitoring.

As shown in Figure 14, the potential workflow includes visible–thermal image acquisition, cow-head detection, heterogeneous-camera target association, local registration of matched head regions, facial-region recognition, and temperature extraction from the corresponding infrared temperature matrix. Visible-light images provide structural cues for stable localization, whereas infrared images provide thermal information. Therefore, the proposed framework can provide a practical basis for future robotic monitoring of facial thermal regions such as the eyes, muzzle, nostrils, ears, and horns [29,30,31,32,33].

4.3. Limitations and Future Work

Several limitations remain. First, the present study verified cow-head detection, cross-camera association, and edge deployment, but it did not complete the full facial-temperature extraction pipeline. Future work should combine the proposed framework with local visible–thermal registration, facial-region recognition, radiometric temperature extraction, and individual identification.

Second, the present system still depends partly on visible-light image quality. At night or under extremely weak illumination without auxiliary lighting, visible images may lose clear contours, texture details, and facial-region cues, which can affect head detection, target separation, and later visible-to-infrared local registration. Infrared pseudo-colour images are less affected by ambient visible light and can provide stable thermal radiation, but their limited texture is insufficient for independent fine localization of specific facial regions. Thus, the current system is best suited to daytime, indoor lighting, or scenarios with supplementary illumination. Future research should exploit the complementarity between visible structural information and infrared thermal information more deeply to improve night-time and low-light robustness.

5. Conclusions

This work investigated dual-light perception for inspection robots and focused on cow-head detection and cross-modal association in visible-light and infrared pseudo-colour images. A compact YOLO11-AFE detector and an improved Hungarian association algorithm were developed. The conclusions are as follows.

• By introducing ADown, C3k2_FE, and SPPF_ECA, YOLO11-AFE improved cow-head detection in both modalities. Compared with YOLO11n, the visible-light Precision, mAP@0.5, and mAP@0.5:0.95 increased by 4.40, 2.15, and 1.36 percentage points. The corresponding gains on the infrared pseudo-colour subset were 1.65, 0.15, and 1.15 percentage points.
• The detector contained 2.14M parameters and required 5.27 GFLOPs, which are 17.37% and 18.17% lower than YOLO11n. Therefore, the model reduces computational demand while improving accuracy, making it appropriate for real-time cow-head detection on inspection-robot edge hardware.
• The improved Hungarian association algorithm uses both target-box IoU and the Euclidean distance difference relative to the image centre. In the 122 synchronized dual-light test-image pairs, 187 corresponding cow-head pairs were present and all were associated without mismatches or missed matches. These results indicate that the method can provide reliable cross-modal target-level correspondence for subsequent facial-region localization and radiometric temperature extraction.