Quantification of Opercular Pigmentation Changes in Farmed Atlantic Salmon: A Novel Application for Computer Vision in Fish Welfare Assessment

1. Introduction

The aquaculture industry is facing persistently high mortality rates in farmed Atlantic salmon, highlighting the need for improved welfare assessment tools and management practices. In Norway, approximately 70 million salmon were lost during the sea phase of production in 2023, of which 62.8 million were recorded as dead, corresponding to a sea-phase mortality of 16.7%, the highest level reported in recent years (Sommerset et al., 2023; Tvete et al., 2023). These losses are associated with multiple health and welfare challenges in aquaculture production, including delousing-related injuries, complex gill disease, winter ulcers and stress associated with management procedures such as delousing and handling (Overton et al., 2019; Rey et al., 2019; Stien et al., 2020; Keihani et al., 2024). This context has stimulated considerable scholarly interest in the development of advanced welfare indicators for assessing welfare parameters across diverse production systems (Adams et al., 2007; Stien et al., 2020; Volpato et al., 2007; Keihani et al., 2024).

Many welfare indicators currently used in salmon farming are invasive, labor-intensive, or become informative only after a welfare problem has already developed. External indicators such as emaciation, wounds, and scale loss are useful, but they are retrospective in nature and may not provide an early warning signal. Physiological indicators, including plasma cortisol, mucus cortisol, and fecal cortisol metabolites, may offer additional insight, but their collection and interpretation remain challenging in practical aquaculture settings (Keihani et al., 2024). Even when sample collection is considered minimally invasive, handling itself may influence the stress response, and the temporal dynamic of cortisol-related measures can complete interpretation (Cao et al., 2017). These limitations motivate the search for alternative indicators that are less invasive and more easily integrated into imaging-based monitoring approaches.

Melanin-based skin pigmentation may represent one such alternative. In Atlantic salmon and other salmonids, dark spots are formed by the aggregation of chromatophores such as melanophores that store and produce eumelanin and are responsible for their black appearance (Bagnara et al., 2006). Eumelanin has been associated with a wide range of ecological and physiological functions, including photoprotection, camouflage, and communication in vertebrates (Leclercq et al., 2010; C’esarini et al., 1996; Riley et al., 1997; Mackintosh el al., 2001; Roulin et al., 2004; Hoekstra et al., 2006). Previous studies in salmonids have further suggested that melanin-based spot patterns may be associated with individual physiological and behavioral differences. For example, in strains of Atlantic salmon (Salmo salar) and rainbow trout (Oncorhynchus mykiss), individuals with a higher number of spots have been reported to show reduced cortisol responsiveness, faster recovery of feeding after transfer to novel environments, and lower ectoparasitic sea lice burdens (Kittilsen et al., 2009, 2012; Khan et al., 2016).

More broadly, body coloration in fish can reflect both long-term and short-term physiological states and changes associated with behavior and environment. Coloration has been linked to social status, reproductive signaling, and agonistic interactions, and can also change in response to environment background for camouflage (Hoglund et al., 2000; Maan et al., 2004; Yasir et al., 2009). It is known that centrifugal dispersion of melanosomes regulated by melanocyte-stimulating hormone (MSH) makes the fish appear darker, conversely, the centripetal aggregation regulated by Melanin-concentrating hormone (MCH) makes the appearance paler or lighter (Logan et al., 2006; Mills et al., 2009). Higher MCH and higher cortisol levels have been associated with salmon infected with pathogens. Additionally, body coloration changes have been observed; specifically, fish showing greater visual distinctness against black backgrounds and lower visual distinctness against white backgrounds (Yi et al., 2021). However, the physiological pathways involved in color changes of skin-based spots under stress (depicted by high cortisol levels) are not yet well understood.

At the same time, advances in computer vision (Voulodimos et al., 2018) provide new opportunities for quantitative, image-based assessment of external phenotypes in fish. Existing applications in aquaculture have focused primarily on disease detection, lice monitoring, wound identification, fish counting, and individual re-identification based on body patterns (Ahmed et al., 2022; Zhang et al., 2024; Gupta et al., 2022; Banno et al., 2022; Cisar et al., 2021). These studies demonstrate the utility of automated and semi-automated image analysis for detecting visual features in fish, but relatively little work has explored whether similar approaches can be used to quantify temporal changes in melanin-based pigmentation features relevant to welfare research.

In the present study, we investigate whether changes in the grayscale appearance of melanin-based skin spots on the operculum of Atlantic salmon can be quantified from image collected before and after exposure to a confinement episode. To do this, we developed a semi-automated computer vision pipeline that combines operculum detection, spot segmentation, and grayscale-based features extraction. The aim of the study is not to establish a validated stress biomarker, but rather to examine whether this imaging-based methodology can detect measurable temporal changes in opercular spots under the applied experimental conditions. We therefore present this work as a proof-of-principle study that may support future research on non-invasive, image-based welfare assessment in Atlantic salmon.

The main contributions of our work are listed as:

• We apply a computer vision-based methodology to quantify melanin-based skin spots on the operculum of Atlantic salmon (salmo salar).
• We propose a semi-automated methodological pipeline for operculum detection, spot segmentation, and grayscale-based quantification of temporal changes in spot appearance.
• We describe an imaging and annotation workflow that may support future studies of opercular pigmentation dynamics in salmon.

The rest of the paper is divided into 4 main sections; section 2 introduces related work, section 3 explains data collection and the methodology, section 4 explains the results, and finally section 5 highlights the main points of the conducted study.

2. Related Works

Existing studies primarily focus on fish detection, monitoring, and re-identification using skin spot patterns. Therefore, we have divided literature into the aforementioned categories.

2.1. Fish Disease Detection

A pipeline composed of image processing (Chaki et al., 2018), feature engineering (Mohanaiah et al., 2013), and subsequent machine learning techniques (Sindhu et al., 2019) have demonstrated promising potential for disease infected salmon identification (Ahmed et al., 2022). Object detection models (Shetty et al., 2021) used for identifying salmon lice in seawater during the pre-infectious stage, have shown promising results for early lice management and assisting in salmon welfare (Zhang et al., 2024). Convolutional neural networks (Wu et al., 2017) customized architecture for accurately classifying wounds and lice in underwater images of Atlantic salmon while catering to varying illumination conditions with image pre-processing has achieved superior performance (Gupta et al., 2022), outperforming state-of-the-art networks such as VGG16/19 (Simonyan et al., 2014). Hybrid approach (Huang et al., 2023) based on multi-layer fusion (Liang et al., 2019), attention mechanism (Vaswani et al., 2017), and online sequential extreme learning machine (Huang et al., 2005) has shown improved training and performance in identification of a wide range of fish diseases.

2.2. Fish Detection and Monitoring

Automated approach (Banno et al., 2022) based on YOLOv4 (Bochkovskiy et al., 2020) has delivered higher counting accuracy of wild fish in sea cages as compared to manual count underlying the application of object detection models in automated fish monitoring under varying lighting conditions. Lateral line scales in fish are considered an important phenotype in species identification (Mogdans et al., 2012), an automated approach (Yu et al., 2023) utilizing modified version of YOLOv5 (Jocher et al., 2020; Khanam et al., 2024) with spatial attention (Vaswani et al., 2017) has shown promising results in lateral line scales detection and counting. Mortality rate is an important fish welfare indicator in retrospect (Ellis et al., 2012), (Ranjan et al., 2023) has implemented a real-time detection and alert system utilizing edge-computing (Cao et al., 2020), Internet of Things (Li et al., 2015), and an object detection model such as Yolov7 (Wang et al., 2023) for mortality monitoring in fish tanks. A three-staged approach (Al Duhayyim et al., 2022) composed of gaussian mixture model (GMM) (Farnoush et al., 2008), object detectors such as Mask-RCNN (He et al., 2017), and wavelet kernel extreme learning machine (WKELM) (Ding et al., 2016) is presented for fish detection and classification.

2.3. Fish Re-Identification

A study (Cisar et al., 2021) has presented short-term identification approach based on convolutional neural networks (O’Shea et al., 2015) and long-term identification based on Euclidean distance (Levy et al., 2024) utilizing histogram of gradients (HOG) (Lowe et al., 2004) features. Both approaches utilize the body spots of salmon for re-identification of salmon across images taken in different photography sessions. (Bekkozhayeva et al., 2022) utilized salmon body scales as an alternate phenotype to body spots and presented a methodology based on Image processing (morphological techniques (Chaki et al., 2018)) for salmon re-identification. Brook trout re-identification framework (Zhou et al., 2022) consisting of: i) Faster-RCNN (Ren et al., 2015) with ResNet-50 (He et al., 2016) backbone trained on open-source dataset (Lin et al., 2014; Kuznetsova et al., 2020) for region of interest extraction containing brook trout, ii) morphological operations (Chaki et al., 2018) and spectrum based filtering (Chaki et al., 2018) for extracting foreground (brook trout) and the pigmentation patterns respectively, iii) followed by a deep neural network such as ResNet-152 (He et al., 2016) for generating feature vectors, iv) finally cosine similarity measure (Levy et al., 2024) for identification of the same brook trout specimens feature vectors obtained across different images has shown promising results by adopting this multistage framework. In (Shi et al., 2023), authors implemented a training framework based on contrastive learning (Chen et al., 2020) for training ResNet-152 (He et al., 2016) used as an encoder model (Gehring et al., 2016). The model was then used to generate temporally aware features utilizing masu salmon lateral line parr marks. Finally, cosine similarity measure (Levy et al., 2024) for these features was adopted for identification of same salmon in images collected over the span of 2 years.

Our methodology is partially based on fish detection and identification domains, employing object detector and transformer-based segmentation model for operculum region and melanin-based skin spots segmentations within those regions respectively. The objective of this research is to test a methodology for quantifying visual changes in the melanin spots under confinement stress episode. Therefore, in this study, we propose a semi-automated methodology based on computer vision and image processing for confinement episode induced visual changes quantification in melanin-based skin spots on the operculum region of Atlantic salmon.

3. Materials and Methods

In this section, we describe the experimental setup for data collection, the annotation process for identifying operculum regions and spots, and the utility of image augmentation techniques to expand these annotations. We then describe how the models are fine-tuned on these annotations for operculum and spot segmentation, followed by the inference process using these finetuned models to assess visual changes in the spots. The methodology pipeline can be seen in Figure 1.

3.1. Experimental Setup and Dataset

A total of 130 Atlantic salmon from a commercial breeding programme (Aquagen AS, Trondheim, Norway), with a mean age of approximately 1.5 years and body mass ranging between 2 and 10 kg, with equal sex representation (65 males and 65 females), were used in this experiment. The fish were kept in a circular tank (7 m diameter) with dark green coloration and a continuous supply of UV-light-illuminated seawater pumped from a depth of 90 m, maintained at a constant temperature of 8.9 °C, at the Matre Research Station, Institute of Marine Research (IMR), Matredal, Norway.

On the first day, unstressed control fish (n = 8) were captured individually by netting and administered a lethal dose of anesthetic (MS-222; 1 g/L). They were photographed on both sides of the body, and close-up images of the head were also captured under ambient (natural sunlight) lighting using an Olympus Corporation TG-6 camera model with settings set to auto. The remaining fish (n = 122) were sedated and photographed in the same manner as the control fish. Following photography, these fish were transferred to a new tank and subjected to confinement stress overnight (approximately 18 h) by lowering the water level. During confinement, the water depth was approximately 10–15 cm at the edge of the tank. In most fish, the dorsal fins were above the water surface, and the largest individuals were unable to remain upright when attempting to swim near the tank wall.

After overnight confinement stress, fish were captured individually using a net, and groups of five to six fish at a time were placed in a 1 m³ container with seawater, after which they were administered a lethal dose of anesthetic and photographed. A total of 1,040 images were captured, comprising eight images per salmon, and were further categorized into head and body images. In this study, only the 520 head images were utilized, as these clearly captured the operculum region at close range and allowed spots to be distinctly visible. Only operculum spots were selected due to their prominence and suitability for semi-automated methodology.

3.2. Image Annotations and Augmentations

The training of the pretrained models for detection and segmentation of operculum regions and spots was supported by annotating the operculum (Figure 3(right)) and spots regions (Figure 3(left)) in 275 close-up head images using Roboflow’s annotation toolkit (Dwyer et al., 2025). Spots were annotated based on an instance segmentation strategy aiming to handle variations in shape, size, and texture. Moreover, to facilitate both detection and semantic segmentation of spots (Hafiz et al., 2020). The operculum region was annotated using standard polygon tool, similarly, spots on the operculum were annotated with both standard and smart polygon tools (based on Segment Anything Model (SAM) (Kirillov et al., 2023)). A total of 9000 spots (32 spots per fish) and 275 operculum regions were annotated. The annotated operculum regions were split into 70% training (193 images), 20% validation (55 images), and 10% testing (27 images) sets. Geometric and pixel intensity transformations (Xu et al., 2023) based augmentations available in Roboflow were applied to the annotated spots images. The motivation behind applying augmentation was to build a dataset with increased variation in shape, size, and texture of spots. The explanation against the augmentation strategies (Dwyer et al., 2024) used are as follows:

Figure 2. (Top-row) Close-up images of the left side of a salmon specimen taken before (left) and after (right) confinement stress. (Bottom-row) Corresponding images of the right side.

Figure 2. (Top-row) Close-up images of the left side of a salmon specimen taken before (left) and after (right) confinement stress. (Bottom-row) Corresponding images of the right side.

Figure 3. Spots (left) and operculum (right) region in the close-up image annotated with smart and standard polygon tools in Roboflow.

Figure 3. Spots (left) and operculum (right) region in the close-up image annotated with smart and standard polygon tools in Roboflow.

3.2.1. Geometric Transformations

• Reflection: Clockwise, counterclockwise, upside-down reflections were generated with their respective transformation matrices using the following equation:

$$Image\left(x,y\right)=center+\left(\sum _{i=0}^{width}\sum _{j=0}^{height}\left(\left(Image\left(i,j\right)-center\right)*M_{cw\left|ccw\right|updw}\right)\right)$$

(1)

where $center=[{\displaystyle \frac{width}{2}},{\displaystyle \frac{Height}{2}}]$ and $M_{cw}=\left[\begin{array}{cc}0& -1\\ 1& 0\end{array}\right],M_{ccw}=\left[\begin{array}{cc}0& 1\\ -1& 0\end{array}\right],M_{updw}=\left[\begin{array}{cc}-1& 0\\ 0& -1\end{array}\right]$ are clockwise, counterclockwise, and upside-down reflection transformation matrices respectively and image spatial resolution; width=1080, height=1080.

• Rotation: Images were randomly rotated using angles (ϴ) randomly sampled from a given range = [${-15}^{°},{15}^{°}$] using the following transformation equation:

  $$Image\left(x,y\right)=center+\left(\sum _{i=0}^{width}\sum _{j=0}^{height}\left(\left(Image\left(i,j\right)-center\right)*M_{rot}\right)\right)$$

  (2)

  where $M_{rot}=\left[\begin{array}{cc}\mathrm{c}\mathrm{o}\mathrm{s}\left(ϴ\right)& -\mathrm{s}\mathrm{i}\mathrm{n}\left(ϴ\right)\\ \mathrm{s}\mathrm{i}\mathrm{n}\left(ϴ\right)& \mathrm{c}\mathrm{o}\mathrm{s}\left(ϴ\right)\end{array}\right]$ is the rotation transformation matrix.
• Shear: Shearing was applied in both horizontal and vertical directions using angles (ϴ) randomly sampled from a given range= [${-10}^{°},{10}^{°}$]
• Crops: Crops were generated with a randomly sampled zoom factor percentage (zf_perc) within a given range= [0, 10%] using the following transformation:

  $$idx=INT\left({\displaystyle \frac{w*h*{\mathrm{z}\mathrm{f}}_{\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{c}}}{w}}\right)$$

  (3)

  $$Crop=I[idx:w+idx,idx:h+idx]$$

  (4)

  where idx is the zoom factor index along which pixels in horizontal (w) and vertical (h) directions of Image (I) are sampled for the crop.

3.2.2. Pixel Intensity Transformations

• Exposure: Image exposure in both directions was randomly adjusted by randomly sampling a threshold from a given range = [-2%, 2%] using the following transformation equations:
• $I_{LAB\leftarrow }cvtColorLAB\left(I_{RGB}\right)$
• $I_L;I_A;I_B\leftarrow split\left(I_{LAB}\right)$
• $exposure=threshold\mathrm{*}255$
• $I_{L\_adjusted}\pm exposure$
• $I_{LAB}\leftarrow merge(I_{L\_adjusted};IA;IB)$
• $I_{RGB}\leftarrow cvtColor\left(I_{LAB}\right)$

where cvtColor; spit; merge is color space conversion, channel-wise splitting, and channel-wise merging OpenCV (Bradski et al., 2000) functions. $I_{RGB}$ is image in RGB color space (Busin et al., 2008) and $I_{LAB}$is the variant of same image in LAB color space (Busin et al., 2008).

• Brightness: Brightness adjustments were like exposure adjustments except instead of utilizing the LAB color space, the RGB images were converted to HSV color space (Busin et al., 2008), and the value channel of the converted images were adjusted.
• Blur and Noise: Blur was introduced using a gaussian filter (Bradski et al., 2000; Nelsonet al., 2020) while for noise salt-and-pepper noise (Bradski et al., 2000; Azzeh et al., 2018) was used.

The augmentations (Figure 4) were applied randomly to training images. Geometric transformations such as reflection, rotation, and shearing render the model invariant to camera orientation related variations while cropping improves the model resilience to variations in size and positioning of the spots and operculum regions. Pixel intensity transformations such as exposure, brightness, and blur were used to improve model robustness to varying lighting conditions and resilience to camera focus. Lastly, noise was added to help the model against adversarial attacks (Goodfellow et al., 2014). The augmented spots training set consisted of 1065 images (approximately representing 35,000 spots) while the validation and test sets were left un-augmented. These augmentations were applied to improve the model robustness to variation in spots characteristics (size, pixel intensities, structure, texture, etc)

3.3. Detection and Segmentation of the Operculum Region and Spots

YOLOv8 (Sohan et al., 2024) and Segment anything 2 (SAM 2) (Ravi et al., 2024) object detection and segmentation models were used for segmentation of operculum regions and spots within those regions respectively. YOLOv8 is an object detector comprising of three primary components: i) a backbone ii) a neck, iii) and a head. i) The backbone includes convolutional blocks, C2f blocks, and a spatial pyramid pooling block. Convolutional block consists of a convolutional layer (Wu et al., 2017), batch normalization layer (Ramachandran et al., 2017), and SiLU activation layer (Yoon et al., 2025). C2f block is further composed of a convolutional block, channel-wise feature maps splitting, bottleneck blocks, feature maps concatenation and finally a convolutional block. The bottleneck block is made up of convolutional blocks with residual connections (He et al., 2016). Spatial pyramid pooling block (He et al., 2015) consists of a convolutional block, maxpooling layers (Wu et al., 2017), residual connections, and finally feature maps concatenation followed by a convolutional block. ii) The Neck component is composed of convolutional blocks, c2f blocks, concatenation layers, and transposed convolutional layers (Zeiler et al., 2010) for upsampling the spatial resolution of the feature maps. iii) The head is made up of convolutional blocks and convolutional layers. C2f blocks enable residual learning which helps mitigate the vanishing gradient problem (He et al., 2016), it is also responsible for learning a hierarchical representation of the data that captures both textural and semantic information. The spatial pyramid pooling block is responsible for learning multi-scale features which renders the model invariant to different sized and scaled objects. The neck is responsible for further refining the multi-scale features for better detections. Finally, the head component is responsible for predicting bounding box coordinates (Ren et al., 2024), segmentation masks (Ren et al., 2024), and class labels (Ren et al., 2024) against the detections. State-of-the-art segmentation model, segment anything 2 (SAM 2) is composed of the following architectural components: i) image encoder, ii) memory attention, iii) mask decoder, iv) prompt encoder, v) memory encoder, and vi) memory bank. i) Image encoder is based on Hiera image encoder (Ryali et al., 2023), which is mainly composed of 4 hierarchical vision transformers (ViT) (Dosovitskiy et al., 2020) termed as 4 stages of the architecture. Two of those stages feature maps are fused using a feature pyramid network (Lin et al., 2017) to produce embeddings against an input image. ii) Memory attention block is made up of a stack of transformers (Turner et al., 2023) responsible for self-attention (Shaw et al., 2018) and follow-up cross-attention (Gheini et al., 2021) with memory embeddings. iii) Prompt encoder follows (Kirillov et al., 2023) encoder enabling prompts through clicks, bounding boxes, and masks. iv) Mask decoder architecture also largely follows (Kirillov et al., 2023) decoder with its bi-directional transformer blocks approach to prompt self-attention and cross attention between prompt-to-image embedding and vice versa. v) Memory encoder reuses the image embedding generated by the image encoder and fuse it with downsampled version of previously generated mask to generate a memory. vi) Memory bank is based on first-in-first-out (FIFO) queue of previously generated memories (including images (frames) and prompts) against previously predicted objects. Image encoder generates image embeddings for a given image which is conditioned on previous frames and their respective predictions by the memory attention block. Mask decoder takes prompt information along with information provided by the memory attention block for generating a prediction for a given image.

3.3.1. YOLOv8 and SAM2.1 Training

Pretrained Yolov8 (Lin et al., 2014) for operculum region segmentation was retrained on non-augmented operculum images dataset with Ultralytics (Jocher et al., 2020) framework. During retraining spatial resolution of 1080x1080 was adopted with a batch size of 4. The whole training duration lasted 100 epochs with training epochs set to 100 with 25 early stopping rounds for mitigating overfitting (Li et al., 2021) along with dropout rate of 20%. Adam optimizer (Adam et al., 2014) with a learning rate of 0.002 and momentum of 0.99 was used for fine-tuning the model. The default data augmentations via the albumentations library (Buslaev et al., 2020) such as Gaussian blur, CLAHE, grayscale conversion, and an 8×8 tile-size adjustment were used during fine-tuning. The comprehensive list of other data augmentations applied during training of the model can be seen in Table 1. The model was evaluated on a set aside validation set with metrics such as recall (Padilla et al., 2021), precision (Padilla et al., 2021) and mean-average precision (mAP) (Padilla et al., 2021). The training and validation losses of the model can be seen in Figure 5 with segmentation loss exhibiting underfitting which could be the consequence of using 20% dropout rate. SAM2.1 (Pretrained Hiera-B+ (Ryali et al., 2023)) was finetuned with augmented dataset of spots with scripts provided by Roboflow (Gallagher, 2020) for 40 epochs. Precision (equation 5), Recall (equation 6), and Mean-Average Precision (equation 7) metrics of YOLOv8 are mentioned in Table 2. Binary cross-entropy (equation 8) and intersection-over-union (equation 9), and mask losses average over 40 epochs of SAM model are mentioned in Table 3. Figure 6 represents the different training losses of SAM model.

$$Recall={\displaystyle \frac{TP}{TP+FP}}$$

(5)

$$Precision={\displaystyle \frac{TP}{TP+FN}}$$

(6)

$$mean-averagePrecision={\displaystyle \frac{1}{C}}\sum _{i=1}^C\sum _{t=1}^T({Recall}_t-{Recall}_{t-1}){Precision}_t$$

(7)

where TP; FP; FN are true positives, false positives, and false negatives. C is the total number of classes (in our case 1, roi-region of interest (operculum)) and T is the threshold on which the recall and precision are computed (Typically set to 0.25).

$$BinaryCrossentropy\left(BCE\right)Loss=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N{[y}_i\log \left(p_i\right)+\left(1-y_i\right)\log (1-p_i)]$$

(8)

$$Intersection-over-Union\left(IoU\right)Loss={\displaystyle \frac{1}{N}}\sum _{i=1}^N(1-{\displaystyle \frac{Bbo{x}_{g{t}_1}\cap Bbo{x}_{pre{d}_i}}{Bbo{x}_{g{t}_1}\cup Bbo{x}_{pre{d}_i}}})$$

(9)

where $\mathrm{y},\mathrm{p}$are the ground truth mask class labels and predicted probabilities against observations belonging to a class. N is total number of observations. ${Bbox}_{pred},Bbo{x}_{gt}$are predicted and ground-truth bounding boxes.

3.4. Inference

At inference, an unseen dataset containing 84 images which were collected from 21 stressed (treated) salmon specimens with each one photographed 4 times (left side: pre, post-stress; right side: pre, post-stress). Operculum regions from these images were extracted through binary masks obtained from the trained YOLOv8 polygon coordinates (Figure 7).

3.4.1. Operculum Regions Registration

The extracted operculum regions were sorted into pre-and post-stress groups and registered to ensure some degree of 1-to-1 correspondence between them for pre-and post-stress spot visual changes analysis later. The image registration pipeline is based on the OpenCV library (Bradski et al., 2000): i) Pre-and post-stress regions sidewise (left and right) are converted to grayscale, ii) Scale-invariant feature transform (SIFT) (Lowe et al., 2004) feature detector is used to detect and compute the feature keypoints and descriptors respectively in the pre-and post-stress grayscale regions, iii) Fast library for approximate nearest neighbors (FLANN) (Muja et al., 2009) based matcher is used to match the feature descriptors, iv) Lowe’s ratio test (Lowe et al., 2004) is applied to sample the best matches for corresponding keypoints extraction, v) the extracted keypoints with RANSAC (Fischler et al., 1981) are used to compute the homography matrix (Dubrofsky et al., 2009), vi) The matrix is applied to the pre-stress region and aligned with the post-stress region. The different stages of registration pipeline can be seen in Figure 8.

3.4.2. Operculum Regions Normalization

The unseen dataset was photographed in ambient lighting conditions similar to models training and evaluation datasets. The registered regions were normalized with white patch retinex algorithm (Ramanath et al., 2014): i) The eye and ID-tag regions of the salmon specimens were manually annotated with the smart polygon tool available in Roboflow (Dwyer et al., 2025), ii) Black pixels in segmented eye regions were sorted in ascending order and 20% of pixels were sampled, averaged, and used as reference black pixels, iii) White pixels in segmented ID-tag regions were sorted in descending order and 20% of pixels were sampled, averaged, and used as reference white pixels, iv) Each RGB salmon specimen operculum region image was converted to LAB color space, the luminance (L) channel was extracted, bias correction was applied using averaged black reference pixels while contrast was adjusted with averaged white reference pixels, v) Furthermore, the pixels in L-channel were clipped between 0 and 255, vi) Finally, the normalized L-channel of the image was merged back with the left out channels (i.e A and B) and converted back to RGB color space. The normalization algorithm output can be seen in Figure 9.

3.4.3. Operculum Region Spots Segmentation

Registered and normalized operculum region images were used to obtain spots localization information (bounding boxes, masks) from the fine-tuned SAM model. Spots (Figure 10) exhibiting specular highlights, complete occlusion due to mucus, and partial occlusion due to water droplets were removed from both pre-and post-stress operculum regions in Roboflow. Moreover, misaligned spot contours and missed spots were also manually corrected and annotated, respectively. The 676 uniquely identified spots fit for grayscale pixel intensity based spot-wise analysis were matched in both pre-and post-stress images using the following steps (Figure 11): i) Center point of each bounding box localizing the spot was computed, ii) Euclidian distances between pre-stress and post-stress spots were computed (in 1-to-many association), iii) Minimum distance between pairs of pre-and post-stress spots were computed and their corresponding labels (segmentation coordinates) were sorted accordingly, iv) Some of the spots with inaccuracies were handled manually for matching.

3.4.4. Feature Extraction

Matched and corrected spot Segmentation polygon coordinates were utilized in constructing binary spot masks and were further utilized in feature extraction using the following steps: i) Operculum images were converted to grayscale images, ii) Segmentation coordinates corresponding to spots in the operculum were used to construct spot binary masks, iii) Grayscale pixels localized by binary masks were sorted into 10 bins of size 0.1, iv) Individual spot pixel intensities in their respective bins as well as intensity means computed from all bins were associated with the respective salmon specimens.

3.4.5. Statistical Analysis (Grayscale Intensity)

Grayscale pixel intensity was analyzed as a repeated-measure outcome in individual fish, with measurements taken on two days and on both the left and right sides. Since the grayscale intensity is a continuous variable bounded between 0 and 1, and because repeated observations within fish are not independent, the data were modeled using a beta-mixed-effects model fit by maximum likelihood. The model included DAY, SIDE, and their interaction as fixed effects, with fish identity as a random intercept to account for within-fish correlation. The mean grayscale intensity increased from Day1 to Day 2 on both left (0.178 to 0.266) and right sides (0.189 to 0.267). Consistent with this pattern, there was a significant effect of Day ($\beta =0.514,SE=0.101,z=5.15,p-value=2.66E-07(p<0.001),95\%CI0.320-0.714$), indicating higher grayscale intensity at Day 2, whereas neither side ($\beta =0.0585,p=0.582$) nor the Day x Side interaction ($\beta =-0.0653,p=0.644$) was statistically significant. Model-based estimated marginal means similarly showed higher grayscale intensity at Day 2 for both sides. Across paired observations, the mean absolute increase in grayscale intensity was 0.083 ($95\%CI0.061-0.105$), corresponding to a mean relative increase of 50.5% ($95\%CI36.5-64.4$). Together, these results indicate a clear temporal increase in grayscale intensity, with no evidence that the magnitude of change differed between the left and right sides. The likelihood-based mixed-model framework is appropriate for bounded repeated-measure data, the modest sample size and variability in individual-level change should be considered when interpreting the strength and generalizability of these findings. The individual trajectories and the boxplot of the grayscale intensities grouped by side and day can be seen in Figure 12 and Figure 13 respectively.

4. Results

4.1. Effect of Treatment on the Spot Pixel Intensities

The grayscale pixel intensities of the spots on Day 1 and Day 2 were averaged separately, and the change in mean pre-stress and post-stress spot pixel intensity in the operculum region was calculated using the following equation:

$$Change={\displaystyle \frac{MeanPixelIntensity\left(Day2\right)-MeanPixelIntensity\left(Day1\right)}{MeanPixelIntensity\left(Day1\right)}}$$

(10)

As shown in Table 4, the response to treatment was heterogeneous across individuals. Several fish showed moderate increases in spot pixel intensity, whereas others showed very large increases, and a few showed negative changes, indicating reduced post-stress intensity relative to pre-stress values. The mean percentage change was positive on both sides, suggesting an overall treatment-associated increase in spot pixel intensity, the widespread values points to considerable inter-individual variation.

4.2. Grayscale Intensities of Treated and Control Groups

The spots sampled from 8 control fish and 21 treated fish were analyzed for outliers removal. The lower bound and upper bound of the extracted features were computed using the following steps: i) the Interquartile range (IQR) was computed utilizing the Q1 (25th percentile) and Q3 (75th percentile) of the data, ii) the lower (Q1—factor * IQR) and upper (Q3—factor * IQR) bounds were computed, with a factor of 1.5, iii) feature values falling below the lower bound or above the upper bound were identified and removed, iv) this process was repeated for 10 iterations to remove all of the outliers from the grayscale intensities. After the removal of outliers, the remaining grayscale intensities were plotted. We can observe in Figure 14 that the distribution of control and pre-stress spots exhibited a similar spread, with an interquartile range (IQR) of approximately 0.12. In contrast, increased variation is observed in the post-stress spots, indicating greater dispersion in grayscale intensity values. We can also observe that the mean value on Day 2 is higher than Day 1, indicating that on average the spots are becoming brighter on Day 2.

4.3. Neighborhood Based Grayscale Intensity Analysis

Grayscale pixel intensities of spot and its neighbors were analyzed on Day 1 (pre-stress) and Day 2 (post-stress) using the following steps: i) The center point of bounding boxes localizing spots were used to compute the Euclidean distances between a spot (source) and the remaining spots (targets) in each operculum region with respect to side and day (1: pre-stress, 2:post-stress), ii) Distances between source and target spots were sorted in ascending order and 4-neighbors per spots were extracted, iii) Coordinates of the source and neighboring spots were used to construct directed graph visualizations (Figure 15(a) (see the supplementary material)) for each operculum-day-side combination, following the tutorials provided by (Hagberg et al., 2008), iv) The differences in pixel intensities between source and target spots for each operculum-day combination were formulated in source wise 1-D vectors, v) Normalized-L1 distances (0:Similar, 1:Completely different) (Levy et al., 2024) between source vectors on day 1 and day 2 were computed. These normalized L1-distances were used to construct boxplots (Figure 15(b) (see the supplementary material)) for each salmon specimen. The variation observed within each specimen reflects the inter-individual differences in changes in spot intensities relative to their neighboring spots on day 2.

4.4. Manual Scoring of Spots on a Grayscale by Observers

A composite-image generation pipeline was developed to visualize annotated spot regions across matched pre-and post-stress image sets. For each fish sample, four (left day 1 and 2, right day 1 and 2) grayscale images were loaded together with their polygon-based spot annotations. A parameter (nspots) was defined to include all annotated spots from each grayscale image, such that the full set of available spot regions in each image was incorporated into mask generation. Spot annotation coordinates, originally represented in normalized form, were transformed into pixel-space coordinates using the dimensions (height,width) of the corresponding source image. These polygons were subsequently rasterized to produce binary masks for each of the four images. To ensure adequate spots region representation, the resulting masks were evaluated against a minimum pixel-area criterion before inclusion in the final visualizations. The four grayscale images and their corresponding binary masks were then assembled into a 2x2 stitched images. From these composites, a stitched green-background spot-placeholder image was rendered, with all annotated spots from the four images for each fish sample represented individually. The pre-stress spots on a green background can be seen in Figure 16.

The stitched images were printed on A4 sheets and two observers manually scored the spots on a grayscale (Figure 17). It can be observed in Table 4 (see the supplementary material) and 5 that manual and algorithm-driven grayscale change scores showed similar central tendencies on the left side, with mean values of 54.83 and 52.69 for the two observers and 54.5 for the algorithm. Agreement between the algorithm and manual scoring was moderate for left-sided spots, indicating that the automated method broadly captured the direction and magnitude of observer-assessed changes in the pre-and post-stress spots. In contrast, right-sided scores were substantially more variable, with wider ranges of values in the manual assessments. Although the algorithmic mean right-side value was 46.5 and the two manual means (52.52 and 65.15), concordance with manual scoring was weak. The algorithm appeared to compress the range of scores, tending to overestimate low values and underestimate extreme values relative to human observers.

Furthermore, to reduce inter-observer variability, grayscale absolute values from the left and right sides were averaged for each observer and compared both between observers and against the machine-derived grayscale values averaged across sides using Pearson correlation analysis.

Pearson correlation analysis showed a strong positive association between Observe E and Observer H measurements (r=0.864, p < 0.001), indicating strong inter-observer agreement. A stronger positive association was observed between the averaged observer measurements and machine-derived values (r=0.966, p < 0.001), suggesting that the machine measurements closely matched the consensus manual assessment. These relationships were consistent across Day 1 and Day 2. The scatterplots for both analyses can be seen in Figure 18.

Table 5. Percentage change in grayscale pixel intensity for each fish and side, as assessed manually by the observers.

Fish Sample	Observer (E)		Observer (H)
	Change left (%)	Change right (%)	Change left (%)	Change right (%)
10	4.5	156.07	13.44	125.44
11	45.14	96.34	16.80	59.44
12	23.12	32.25	-5.66	31.19
13	36.5	3.41	65.47	-8.90
14	86.5	41.95	32.01	37.03
15	11.37	5.22	41.66	0.87
16	12.8	43.08	18.32	4.26
17	59.5	51.11	76	67.64
18	118.7	132.83	75	104.71
19	76.9	0	127.27	0
20	66.36	14.47	37.85	0
21	65.16	91.30	133.54	128.08
22	41.24	164.72	23.32	97.76
23	N/A	N/A	N/A	N/A
24	57.47	61.29	25.31	94.66
25	N/A	N/A	N/A	N/A
26	40.84	96.55	65.76	72
27	86.82	235.08	56.52	110
28	36.58	-4.54	36.60	63.20
29	59.01	0	28.04	0
30	113	16.66	133.91	9.68
Average	54.83	65.15	52.69	52.52

Table 5. Percentage change in grayscale pixel intensity for each fish and side, as assessed manually by the observers.

Fish Sample	Observer (E)		Observer (H)
	Change left (%)	Change right (%)	Change left (%)	Change right (%)
10	4.5	156.07	13.44	125.44
11	45.14	96.34	16.80	59.44
12	23.12	32.25	-5.66	31.19
13	36.5	3.41	65.47	-8.90
14	86.5	41.95	32.01	37.03
15	11.37	5.22	41.66	0.87
16	12.8	43.08	18.32	4.26
17	59.5	51.11	76	67.64
18	118.7	132.83	75	104.71
19	76.9	0	127.27	0
20	66.36	14.47	37.85	0
21	65.16	91.30	133.54	128.08
22	41.24	164.72	23.32	97.76
23	N/A	N/A	N/A	N/A
24	57.47	61.29	25.31	94.66
25	N/A	N/A	N/A	N/A
26	40.84	96.55	65.76	72
27	86.82	235.08	56.52	110
28	36.58	-4.54	36.60	63.20
29	59.01	0	28.04	0
30	113	16.66	133.91	9.68
Average	54.83	65.15	52.69	52.52

5. Discussion and Future Research

The main objective of this study was to investigate whether changes in the grayscale intensity of melanin-based skin spots on the operculum of Atlantic salmon could be detected using a semi-automated computer vision pipeline. The results indicate that such changes can be quantified from out-of-water images collected before and after the applied confinement episode. Across the analyzed fish, the mean grayscale intensity of opercular spots increased from Day 1 to Day 2, and the mixed-effects analysis supported a significant temporal effect. At the same time, responses varied substantially among individuals, indicating considerable heterogeneity in the magnitude and direction of change.

The findings suggest that the proposed methodology is sensitive to changes in spot appearance under the present experimental conditions. The confinement challenge was intended to elicit a stress response, but no independent physiological stress markers such as plasma, mucus, or fecal cortisol were reported in this study to validate the relationship between spot appearance and stress status (Keihani et al., 2024; Cao et al., 2017). In addition, the applied challenge may have involved confounding factors that could also have influenced pigmentation, including altered brightness conditions, body positioning, and context-dependent color responses. Fish coloration is known to vary for multiple reasons, including background adaptation, social signaling, and physiological state (Hoglund et al., 2000; Yasir et al., 2009; Leclercq et al., 2010; Yi et al., 2021). The present design does not fully separate these possible drivers. For this reason, the changes observed in this study should be interpreted as treatment-associated rather than as definitive evidence of a validated stress-specific response. Furthermore, another limitation concerns the control group. Although control fish were included, the sample size was small relative to the treated group, and the comparison between the two groups should therefore be interpreted cautiously. The present study is therefore best understood as a proof-of-principle investigation showing that opercular melanin-based spots appearances can change measurably across repeated imaging points under the applied confinement episode conditions.

Methodologically, the study demonstrates that semi-automated segmentation of the operculum and spot regions is feasible with high accuracy. The trained YOLOv8 model performed strongly for operculum segmentation, and the SAM-based approach enabled detailed spot extraction within the opercular region. These components were essential for constructing a pipeline capable of spot-level quantification. Similar developments in computer vision have already shown considerable promise in aquaculture applications, including wound detection, lice detection, fish monitoring, and pattern-based identification (Gupta et al., 2022; Zhang et al., 2024; Banno et al., 2022; Cisar et al., 2021; Ahmed et al., 2022; Zhou et al., 2022). The manual correction steps required during inference nevertheless highlight that the workflow is not yet fully automated. Occlusion caused by mucus, water droplets, and reflections still required manual intervention, which limits current scalability and real-time applicability.

Another important consideration is the sensitivity of the entire workflow to imaging conditions. Because the analysis relies on grayscale intensity, any variation in lighting, exposure, focus, viewing angle, or reflective artifacts may influence the extracted features. To reduce this problem, we applied image normalization and visually screened spots for suitability prior to analysis. We also restricted the study to opercular spots, as these were generally more prominent and more consistently visible than spots in other body regions. Despite these precautions, residual variation related to image acquisition cannot be excluded. The weaker right-side agreement suggests that some aspects of image registration, segmentation consistency, or visual scoring remain insufficiently controlled. At the same time, the positive correlation between the observers and between the observers and the machine-derived grayscale values suggest that the algorithm is generally able to track the direction of change and shows overall agreement with manual assessment.

The neighborhood-based analysis provided additional descriptive information by examining changes in spot intensity relative to neighboring spots within the same operculum. The variability observed within and among fish indicates that the response was not spatially uniform. This may reflect local pigmentation dynamics, technical variability in image acquisition and matching, or a combination of both. From a biological perspective, this interpretation remains tentative. Previous work has shown that melanin-based pigmentation in salmonids may be associated with differences in stress responsiveness and other aspects of individual phenotype (Kittilsen et al., 2009, 2012; Khan et al., 2016), but the mechanisms underlying short-term visual changes in discrete opercular spots are still not completely understood. Although the observed spatial heterogeneity is intriguing, the present study was not designed to resolve the biological basis of localized spot change.

Overall, the results support the view that image-based quantification of opercular spot appearance may be useful for welfare-related research in Atlantic salmon. The study does not establish opercular spot intensity as a validated biomarker for stress, but it does show that measurable changes in melanin-based spot appearance can be captured using computer vision methods under a defined experimental setup. In that sense, the work provides a methodological foundation for further studies on pigmentation dynamics in salmon.

In the future, we will focus on experimental validation and rigorous control of image acquisition conditions. This includes standardized lighting, camera settings, camera-to-subject distance, and fish positioning. Future experiments will also include larger and more balanced control groups, along with confinement conditions that minimize confounding effects from background adaptation and other coloration responses. In addition, the biological relevance of spot changes validated against established physiological stress indicators, such as cortisol and other endocrine or neurochemical markers. Finally, testing the method on more diverse datasets spanning multiple stocks, environments, and pigmentation phenotypes to access its generalizability and robustness.

In summary, this study should be viewed as proof-of-principle demonstration that melanin-based opercular spots in Atlantic salmon can be detected, segmented, and quantitatively analyzed using computer vision, and that their grayscale appearance may change following exposure to confinement episode. While the findings are promising, further validation is required before such changes can be interpreted confidently as a stress-specific welfare indicator or translated into practical monitoring applications.

6. Conclusion

This study provides a proof-of-principle demonstration that computer vision can be used to detect, segment, and quantify melanin-based opercular spots and treated associated changes in their grayscale intensities in Atlantic salmon. The observed changes in grayscale intensity between pre-and post-confinement images show that the method can capture temporal changes in spot appearance under present experimental conditions. However, the findings should be interpreted cautiously, as stress was not independently validated and alternative causes of color change were not excluded in this study. Our work establishes a methodological basis for future studies on opercular pigmentation as a potential non-invasive welfare related indicator.