LLMSeamCarver: LLM-Enhanced Content-Aware Image Resizing

I. Introduction

LLMSeamCarver is an image resizing method designed to maintain important image details during resizing. By integrating large language models (LLMs), LLMSeamCarver goes beyond traditional methods, enabling a smarter, context-aware resizing process. LLMs enhance tasks such as region prioritization, interpolation, edge detection, and energy calculation by analyzing image context or textual inputs. This allows LLMSeamCarver to preserve crucial image features like faces and text while optimizing resizing for different scenarios.

This paper explores how LLMs improve the accuracy and efficiency of image resizing.

II. Background

i. Image Resizing and LLMs

SeamCarver enhances the traditional seam carving method by incorporating LLMs. The integration of LLMs allows for dynamic resizing of images based on real-time context and user input, rather than relying on fixed parameters. The LLM dynamically optimizes the energy functions that determine how seams are selected and removed, making the resizing process adaptive and context-sensitive. This approach allows for superior preservation of image details and better quality when resizing for specific tasks, such as creating thumbnails or preparing images for various screen sizes.

LLMs contribute by adjusting the parameters based on the content of the image and the desired effect, improving the quality of resized images while providing flexibility. The dynamic optimization of the resizing process using LLMs represents a major leap in the flexibility and efficiency of image resizing techniques.

ii. LLM-Related Work

Recent research in Large Language Models (LLMs) has demonstrated their potential in a variety of domains, including text, image, and video generation.

Recently, LLMs has shown its power and potential to enhance traditional image processing workflows through advanced model architectures and optimization techniques. For instance, Li et al. (2024) demonstrated how LLMs can optimize convolutional neural network (CNN) layers for feature extraction in image processing tasks, improving the performance of deep learning models in image classification and segmentation [1]. Zhang et al. (2024) explored the use of LLMs in multi-modal fusion networks, enabling the integration of both visual and textual information, which enhances image analysis tasks [2,3,4,5].

Additionally, efficient algorithm design [5,6,7,8,9,10,11,12] and efficient LLMs [13,14] have shown promising prospective in efficient model design with LLMs. Through dynamic optimization, LLMs allow for more context-aware resizing by adjusting energy functions during the process. This flexibility ensures that fine-grained details are preserved, which is especially crucial for tasks like content-aware resizing, as seen in techniques such as SeamCarver. Studies on text-to-image models have demonstrated how LLMs can modify images based on contextual prompts [1,15,16,17,18,19,20], providing further advancements in content-aware image resizing.

These advancements highlight the increasing role of LLMs in enhancing traditional image processing tasks and their ability to contribute significantly to content-aware resizing techniques like SeamCarver.

iii. Image and Vision Generation Work

The application of deep learning techniques in image and vision generation has also seen significant advancements [4,11,21,22,23,24,25,26]. Deep convolutional networks, for instance, have been used for texture classification (Bu et al., 2019), which is directly relevant to tasks like energy function optimization in SeamCarver [10]. These methods help improve the detail preservation during image resizing by ensuring that textures and edges are maintained throughout the process.

Furthermore, multi-modal fusion networks and techniques for image-driven predictions (as demonstrated by Dan et al., 2024) offer important insights into how AI can be used to process and modify images in real-time [11,21]. Besides, model compression currently becoming a favor from both model and system design perspective [27,28]. These innovations align closely with SeamCarver’s goal of dynamic, user-controlled image resizing, making them valuable for future developments in image resizing technology.

iv. Image Resizing and Seam Carving Research

SeamCarver builds upon earlier work in image resizing techniques. In addition to the foundational work by Avidan and Shamir (2007), other studies have contributed to enhancing seam carving methods. Kiess (2014) introduced improved edge preservation methods within seam carving [29], which is crucial for ensuring that resized images do not suffer from visible distortions along object boundaries. Zhang (2015) compared classic image resizing methods and found that seam carving provided superior results when compared to simpler resizing techniques, particularly in terms of detail preservation [30].

Frankovich (2011) further advanced seam carving by integrating energy gradient functionals to enhance the carving process, providing even more control over the resizing operation [31]. These improvements are incorporated into SeamCarver, which leverages LLMs to further optimize the parameter tuning and energy functions during resizing.

v. Impact of SeamCarver and Future Directions

The development of SeamCarver represents a significant step forward in content-aware image resizing. By leveraging the power of LLMs, this approach enables adaptive resizing, maintaining high-quality images across a variety of use cases. As machine learning and AI continue to evolve, future versions of SeamCarver could integrate even more advanced techniques, such as generative models for even higher-quality resizing and multi-task learning to tailor resizing for specific contexts.

Moreover, SeamCarver provides an excellent example of how LLMs can be used to enhance traditional image processing tasks, enabling more intelligent and user-driven modifications to images. This work will likely spur further research into dynamic image resizing and contribute to more versatile, AI-enhanced image editing tools in the future.

III. Functionality

SeamCarver leverages LLM-Augmented methods to ensure adaptive, high-quality image resizing while preserving both structural and semantic integrity. The key functionalities are:

• LLM-Augmented Region Prioritization: LLMs analyze semantics or textual inputs to prioritize key regions, ensuring critical areas (e.g., faces, text) are preserved.
• LLM-Augmented Bicubic Interpolation: LLMs optimize bicubic interpolation for high-quality enlargements, adjusting parameters based on context or user input.
• LLM-Augmented LC Algorithm: LLMs adapt the LC algorithm by adjusting weights, ensuring the preservation of important image features during resizing.
• LLM-Augmented Canny Edge Detection: LLMs guide Canny edge detection to refine boundaries, enhancing clarity and accuracy based on contextual analysis.
• LLM-Augmented Hough Transformation: LLMs strengthen the Hough transformation, detecting structural lines and ensuring the preservation of geometric features.
• LLM-Augmented Absolute Energy Function: LLMs dynamically adjust energy maps to improve seam selection for more precise resizing.
• LLM-Augmented Dual Energy Model: LLMs refine energy functions, enhancing flexibility and ensuring effective seam carving across various use cases.
• LLM-Augmented Performance Evaluation: CNN-based classification experiments on CIFAR-10 are enhanced with LLM feedback to fine-tune resizing results.

IV. LLM-Guided Region Prioritization

To enhance the seam carving process, we integrate Large Language Models (LLMs) to guide region prioritization during image resizing. Traditional seam carving typically removes seams based on an energy map derived from pixel-level intensity or gradient differences. However, this method may struggle to preserve regions with semantic significance, such as faces, text, or objects, which require more context-aware resizing. Our approach introduces LLMs to assign semantic importance to different regions of the image, modifying the energy map to prioritize the preservation of these crucial regions.

i. Method Overview

Given an image I, the initial energy map $E(x,y)$ is computed using standard seam carving techniques, typically relying on pixel-based features such as intensity gradients and contrast:

$$E(x,y)=\sqrt{{(\nabla I_x(x,y))}^2+{(\nabla I_y(x,y))}^2}$$

where $\nabla I_x(x,y)$ and $\nabla I_y(x,y)$ represent the gradient values of the image I at pixel $(x,y)$ in the x- and y-directions, respectively.

Next, a Large Language Model (LLM) is employed to analyze either the image content directly or a user-provided textual description of the regions to prioritize. For example, a user might specify that "faces should be preserved" or "text should remain readable." This description is processed by the LLM, which assigns an importance score $S(x,y)$ to each pixel based on its semantic relevance. The function that generates these scores is denoted as:

$$S(x,y)=f_{\mathrm{LLM}}(I,D)$$

where $f_{\mathrm{LLM}}$ represents the output of the LLM processing both the image I and a description D. The LLM interprets the description D through its internal knowledge of language and context, identifying which parts of the image correspond to higher-priority regions (e.g., faces, text, objects).

The LLM’s understanding of the image is derived using advanced techniques like **transformer architectures** [32] and **contextual embedding** [33], which allow the model to capture both local and global relationships within the image, ensuring that important features are accurately recognized and prioritized. For example, the LLM might recognize that a region containing a face is more important than a background area when performing resizing.

ii. Energy Map Adjustment

To modify the energy map, the semantic importance scores $S(x,y)$ are combined with the original energy map $E(x,y)$. This modified energy map $E^{\prime }(x,y)$ is calculated as follows:

$$E^{\prime }(x,y)=E(x,y)+\alpha ·S(x,y)$$

where $\alpha$ is a scalar weight that determines the influence of the LLM-based importance scores on the energy map. By incorporating $S(x,y)$, the energy map becomes content-aware, ensuring that the regions with higher semantic importance (e.g., faces, text) have lower energy values, making them less likely to be removed during the seam carving process.

iii. Energy Map Adjustment

The modified energy map $E^{\prime }(x,y)$ is calculated as:

$$E^{\prime }(x,y)=E(x,y)+\alpha S(x,y)$$

where: - $E(x,y)$ is the original energy map. - $S(x,y)$ is the semantic importance score derived from the LLM. - $\alpha$ is a weight factor controlling the influence of semantic importance.

iv. Pseudocode

Algorithm 1:LLM-Guided Region Prioritization for Seam Carving

Initialize: Compute the initial energy map $E(x,y)$ for the image I; Obtain semantic importance scores $S(x,y)$ from LLM based on image content or user description; Normalize the importance scores $S(x,y)$ to a suitable range. Adjustment: 1. For each pixel $(x,y)$, compute the adjusted energy map: $$E^{\prime }(x,y)=E(x,y)+\alpha ·S(x,y)$$ 2. Set $\alpha$ to control the influence of semantic importance on the energy map. 3. Repeat for all pixels to generate the adjusted energy map $E^{\prime }$. Output: The adjusted energy map $E^{\prime }$ for guiding seam carving.

V. LLM-Augmented Bicubic Interpolation

SeamCarver integrate a LLM-Augmented bicubic interpolation method for image resizing. This method uses a bicubic policy to smooth pixel values, with LLMs improving the visual quality of enlarged images. However, traditional bicubic interpolation does not account for the semantic importance of image regions. To address this limitation, we augment the standard interpolation with semantic guidance from LLMs, ensuring that regions of high importance—such as faces, text, and objects—are preserved more effectively during enlargement.

The traditional bicubic interpolation algorithm operates by using a 4x4 pixel grid surrounding the target pixel to calculate the new pixel value. This method typically focuses on the rate of change between neighboring pixel intensities. In contrast, our approach leverages LLMs to assign semantic importance scores $S(x,y)$ to each pixel, reflecting its contextual significance. These importance scores are derived from the image content or a user-provided description, and they adjust the interpolation weights, effectively guiding the resizing process to preserve critical regions.

The bicubic interpolation formula for a pixel at position $(x,y)$ is based on calculating the weighted sum of the 4x4 neighborhood of surrounding pixels. Traditionally, the interpolation weights $w\left(x\right)$ and $w\left(y\right)$ are determined based on the relative distance between the target pixel and its neighbors. These weights can be defined as:

$$w\left(x\right)=\left\{\begin{array}{cc}{(a+2)\left|x\right|}^3-(a+3){\left|x\right|}^2+1,\hfill & \mathrm{if}\left|x\right|\le 1\hfill \\ {a\left|x\right|}^3+{5a\left|x\right|}^2+8a\left|x\right|-4a,\hfill & \mathrm{if}1<\left|x\right|<2\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

Then, the new pixel value at position $(X,Y)$ is computed by summing the contributions of the surrounding 16 pixels:

$$B(X,Y)=\sum _{i=0}^3\sum _{j=0}^3{a}_{ij}·w(x-X_i)·w(y-Y_j)$$

For a floating-point pixel coordinate $(x,y)$, the interpolation involves considering the 4x4 neighborhood $(x_i,y_j)$, where $i,j=0,1,2,3$, and calculating the new pixel value as follows:

$$f(x,y)=\sum _{i=0}^3\sum _{j=0}^{3}f(x_i,y_j)·w(x-x_i)·w(y-y_j)$$

In the augmented version, the weights $w\left(x\right)$ and $w\left(y\right)$ are modified based on the importance scores $S(x,y)$ derived from the LLM. For each pixel, we compute the adjusted interpolation weight $w^{\prime }\left(x\right)$ as:

$$w^{\prime }\left(x\right)=w\left(x\right)·(1+\beta ·S(x,y))$$

where $\beta$ is a scalar factor that controls the influence of the semantic importance score. By incorporating these adjusted weights into the interpolation process, regions deemed important by the LLM receive greater priority during the resizing process, resulting in higher-quality enlargements that better preserve semantic content.

The incorporation of LLMs significantly improves the ability of bicubic interpolation to perform content-aware resizing, ensuring that important regions, such as faces, text, or other key objects, are preserved with higher fidelity. The LLM’s ability to interpret the image context or a user’s textual description enables a more adaptive resizing strategy, where the image can be enlarged in a way that prioritizes and preserves the most semantically relevant regions.

In conclusion, this approach not only enhances the visual quality of enlarged images by preserving important areas but also allows for a more flexible and context-aware image resizing process. The integration of LLMs elevates bicubic interpolation from a purely geometric operation to a more intelligent, context-sensitive method, improving overall resizing performance.

VI. LLM-Augmented LC (Loyalty-Clarity) Policy

SeamCarver also used LLM-Augmented LC (Loyalty-Clarity) Policy to resize images. Traditionally, the LC policy evaluates each pixel’s contrast relative to the entire image, focusing on maintaining the most visually significant elements. However, by incorporating **LLMs**, we enhance this method with semantic understanding, allowing the system to prioritize image regions based not only on visual contrast but also on their semantic importance, as understood from contextual descriptions or image content analysis.

i. Global Contrast Calculation with LLM Influence

The traditional LC policy computes the global contrast of a pixel by summing the distance between the pixel in question and all other pixels in the image. This measure indicates the pixel’s relative importance in terms of visual contrast. In our **LLM-augmented** approach, the global contrast is modified by considering semantic relevance, as dictated by the LLM’s analysis of the image or user-provided description.

For instance, if a user inputs that the image contains important "faces" or "text," the LLM assigns higher weights to these regions, increasing their importance in the contrast calculation. The LLM’s guidance is mathematically integrated into the contrast calculation as follows:

$$SalS\left(I_k\right)=\sum _{\forall I_i\in I}\parallel I_k-I_i\parallel +\lambda ·\sum _{r\in R}{w}_r·\parallel I_k-I_r\parallel$$

(1)

In this formulation: - $I_k$ represents the intensity of the pixel being analyzed, while $I_i$ represents the intensity of all other pixels. - $w_r$ is the weight assigned to a region r by the LLM, which is based on its semantic importance, such as prioritizing faces or text. - $\lambda$ is a scaling factor that controls the influence of the **LLM**’s weighting on the global contrast calculation.

By adjusting $w_r$ based on the LLM-driven understanding of important regions, the algorithm effectively prioritizes preservation of the semantically significant areas.

ii. Frequency-Based Refinement with LLM Augmentation

To further refine the contrast measure, we incorporate the frequency distribution of intensity values in the image. The traditional frequency-based contrast is enhanced with the LLM’s semantic input, which guides how regions of different intensities should be prioritized.

In the standard approach, the global contrast for a pixel $I_k$ is computed as:

$$SalS\left(I_k\right)=\sum _{n=0}^{255}{f}_n\parallel a_m-a_n\parallel$$

(2)

Where: - $f_n$ is the frequency of the intensity value $a_n$. - $a_m$ represents the intensity of the pixel $I_k$, and $a_n$ are the intensity values of all other pixels.

In the LLM-augmented approach, the LLM provides additional weighting for specific regions, emphasizing the importance of certain intensities based on semantic input. The modified calculation is:

$$SalS\left(I_k\right)=\sum _{n=0}^{255}{f}_n\parallel a_m-a_n\parallel +\lambda ·\sum _{r\in R}{w}_r·f_r·\parallel a_m-a_r\parallel$$

(3)

Here, $w_r$ adjusts the weight of the frequency term for pixels in semantically significant regions, as determined by the LLM. This allows for a more refined and context-aware adjustment of the image’s contrast, ensuring that the most relevant image areas are preserved during the resizing process.

iii. Application in Image Resizing

By incorporating LLM-guided adjustments into the LC Policy, SeamCarver becomes significantly more content-aware. The LLM allows the software to prioritize critical regions—such as human faces, text, or objects—based on user input or semantic analysis of the image. This semantic understanding of the image ensures that, even during resizing, key features remain sharp and well-defined, while less important regions are more freely adjusted.

For example, if a user specifies that "faces" should be preserved, the LLM ensures that these areas have a higher weight during the resizing process, while the surrounding less important areas can be resized with minimal distortion. This LLM-augmented LC Policy thus improves the visual integrity of resized images, making the process more adaptable to both user needs and semantic context.

Figure 1. Outlier detected by the LC algorithm

Figure 1. Outlier detected by the LC algorithm

VIII. LLM-Augmented Hough Transformation

The LLMSeamCarver integrates the LLM-Augmented Hough Transformation to enhance line detection during image resizing. This method incorporates semantic guidance from Large Language Models (LLMs) to ensure the preservation of key structural features, such as text or faces, during resizing. The LLM augments the traditional Hough Transformation by adjusting the accumulator based on the semantic importance of regions within the image.

i. Algorithm Overview

The Hough Transformation detects lines by mapping points from the image domain to the Hough space. In the traditional approach, collinear points converge to peaks in Hough space, indicating the presence of a line. The LLM-Augmented Hough Transformation introduces semantic weighting to this process, ensuring that semantically significant lines (e.g., those in text or faces) are prioritized during line detection.

ii. Mathematical Formulation

In the standard Hough Transformation, the accumulator is updated by incrementing for each detected edge pixel. However, in the LLM-Augmented Hough Transformation, the update to the accumulator is influenced by a semantic importance score $S(x,y)$ derived from the LLM. This adjustment ensures that semantically important regions of the image have a greater influence on the detection of lines.

The basic equation for updating the accumulator with the semantic weight $S(x,y)$ is:

$$Accumulator(r,\theta )\leftarrow Accumulator(r,\theta )+S(x,y)$$

(9)

Where: - $r=xcos\left(\theta \right)+ysin\left(\theta \right)$ is the radial distance in Hough space. - $\theta$ is the angle of the line in Hough space. - $S(x,y)$ is the semantic importance score of the pixel $(x,y)$, as determined by the LLM.

This formulation adjusts the weight of each edge pixel based on the semantic relevance of its location in the image. Higher $S(x,y)$ values are assigned to pixels in semantically important regions, such as those in faces or text.

iii. LLM-Augmented Hough Transformation Algorithm

The implementation of the LLM-Augmented Hough Transformation in LLMSeamCarver is outlined as follows:

Algorithm 2:LLM-Augmented Hough Transformation for Line Detection

Require:  $Image$: input digital image; Ensure:  Lines detected in the image, with semantic guidance from the LLM. 1: Apply edge detection (e.g., Canny edge detector) to the image. 2: Initialize Hough space and accumulator: $Accumulator(r,\theta )=0$ 3: for each edge pixel $(x,y)$ in the image do 4:     for each angle $\theta$ do 5:         Compute radial distance $r=xcos\left(\theta \right)+ysin\left(\theta \right)$ 6:         Retrieve the semantic score $S(x,y)$ from the LLM for pixel $(x,y)$ 7:         Update accumulator: $Accumulator(r,\theta )\leftarrow Accumulator(r,\theta )+S(x,y)$ 8:     end for 9: end for 10: Detect peaks in the accumulator. 11: Convert infinite lines to finite lines.

iv. Significance of LLM-Augmented Hough Transformation in Image Resizing

The LLM-Augmented Hough Transformation allows LLMSeamCarver to preserve important linear structures during resizing. By integrating semantic scores from the LLM, the algorithm ensures that critical features, such as text or faces, are prioritized in the resizing process, making the output more content-aware and maintaining the visual integrity of the resized image.

Figure 4. Original Image

Figure 4. Original Image

Figure 5. Lines Detected by LLM-Augmented Hough Transformation with Background

Figure 5. Lines Detected by LLM-Augmented Hough Transformation with Background

Figure 6. Prominent Lines Detected by LLM-Augmented Hough Transformation

Figure 6. Prominent Lines Detected by LLM-Augmented Hough Transformation

IX. LLM-Augmented Absolute Energy Equation

The LLM-Augmented Absolute Energy Equation refines energy calculations by integrating semantic weights $S(x,y)$ derived from Large Language Models (LLMs). This ensures seam carving preserves critical features such as text and faces.

i. Semantic Weighting

The LLM assigns a semantic score $S(x,y)$ to each pixel:

$$S(x,y)=\mathrm{LLM}(I,(x,y),\mathrm{context}),$$

(10)

where I is the input image and context includes features like object and text importance. $S(x,y)\in [0,1]$ scales pixel importance, with higher values for semantically significant regions.

ii. Gradient Refinement

The original energy gradient:

$$e\left(I\right)=\left|{\displaystyle \frac{\partial I}{\partial x}}\right|+\left|{\displaystyle \frac{\partial I}{\partial y}}\right|$$

is modified as:

$$e_{\mathrm{LLM}}\left(I\right)=\left(\left|{\displaystyle \frac{\partial I}{\partial x}}\right|·S_x(x,y)\right)+\left(\left|{\displaystyle \frac{\partial I}{\partial y}}\right|·S_y(x,y)\right),$$

(11)

where $S_x(x,y)$ and $S_y(x,y)$ are direction-specific weights computed by the LLM.

iii. Cumulative Energy Update

The cumulative energy function integrates $S(x,y)$ into the seam carving process:

$$\begin{array}{cc}\hfill e_{\mathrm{LLM}}(i,j)& =e(i,j)+|e(i,j+1)-e(i,j)|·S_x(i,j)\hfill \\ \hfill & +|e(i+1,j)-e(i,j)|·S_y(i,j)\hfill \\ \hfill & +min\{C_L(i,j)+e(i-1,j-1),\hfill \\ \hfill & C_R(i,j)+e(i-1,j+1),\hfill \\ \hfill & C_U(i,j)+e(i-1,j)\}.\hfill \end{array}$$

(12)

Here, $C_L,C_R,C_U$ are adjusted by $S(x,y)$ to prioritize semantically significant pixels.

X. LLM-Augmented Dual Gradient Energy Equation

The proposed LLM-Augmented Dual Gradient Energy Equation utilizes Large Language Models (LLMs) to refine edge detection by dynamically adjusting numerical differentiation and gradient computation. LLMs provide context-aware corrections for each computational step.

i. Numerical Differentiation with LLM Adjustments

Taylor expansions are dynamically adjusted with LLM corrections. The forward expansion is expressed as:

$$f(x+\Delta x)=f\left(x\right)+\Delta x{f}^{\prime }\left(x\right)+{\displaystyle \frac{\Delta x^2}{2}}{f}^{\prime \prime }\left(x\right)+{\Delta }_{LLM},$$

(13)

where ${\Delta }_{LLM}$ includes context-aware corrections predicted by the LLM. Similarly, for the backward expansion:

$$f(x-\Delta x)=f\left(x\right)-\Delta x{f}^{\prime }\left(x\right)+{\displaystyle \frac{\Delta x^2}{2}}{f}^{\prime \prime }\left(x\right)+{\Delta }_{LLM}.$$

(14)

LLMs adapt $\Delta x$ dynamically based on local image gradients and refine higher-order terms to reduce numerical error.

ii. Gradient Approximation with Adaptive Refinements

Gradient approximations in the x- and y-directions incorporate corrections from LLMs:

$$f_x(x,y)\approx {\displaystyle \frac{f(x+\delta ,y)-f(x,y)}{\delta }}+\Delta f_x,$$

(15)

$$f_y(x,y)\approx {\displaystyle \frac{f(x,y+\delta )-f(x,y)}{\delta }}+\Delta f_y.$$

(16)

Here, $\Delta f_x$ and $\Delta f_y$ are LLM-predicted corrections based on local edge strength and texture complexity. The LLM also adapts $\delta$ to handle regions with high-gradient variations.

iii. Energy Calculation with LLM Refinements

The energy of a pixel is computed using the LLM-enhanced gradients for each RGB channel. For the x-direction:

$${\Delta }_x^2(x,y)=\sum _{C\in \{R,G,B\}}{\left(C_x(x,y)+\Delta C_x(x,y)\right)}^2,$$

(17)

and similarly for the y-direction:

$${\Delta }_y^2(x,y)=\sum _{C\in \{R,G,B\}}{\left(C_y(x,y)+\Delta C_y(x,y)\right)}^2.$$

(18)

The total energy is:

$$E(x,y)=\sqrt{{\Delta }_x^2(x,y)+{\Delta }_y^2(x,y)}.$$

(19)

LLM contributions include predicting $\Delta C_x(x,y)$ and $\Delta C_y(x,y)$ to improve accuracy and dynamically adjusting channel weights for better feature preservation.

XI. Result Evaluation

To evaluate the effectiveness of the image resizing methods in LLMSeamCarver, we conducted an experiment using Convolutional Neural Networks (CNNs) for image classification. The goal was to compare how different resizing techniques impact classification accuracy. Additionally, we explored the role of LLM-augmented approaches in enhancing image feature preservation and improving classification outcomes after resizing.

i. Experimental Setup

In the experiment, images were resized using various methods implemented in LLMSeamCarver, including traditional methods and LLM-augmented approaches. The resized images were then fed into a CNN model to assess how well each resizing method preserved image features essential for accurate classification. The CIFAR-10 dataset, a well-known benchmark in image classification, was used for this experiment.

ii. Methodology

The workflow of the experiment is as follows:

Figure 7. Workflow of the experiment, including LLM-augmented methods for image resizing

Figure 7. Workflow of the experiment, including LLM-augmented methods for image resizing

The CNN model was first trained on the original CIFAR-10 images, and subsequently, the same model was used to classify images that had been resized using different methods in LLMSeamCarver. This allowed us to evaluate how each resizing method influenced the model’s ability to recognize key features. Additionally, LLM-augmented methods were used to improve the preservation of important image details during resizing, enhancing classification accuracy.

iii. Results and Discussion

The results, including accuracy metrics and error rates for each sub-experiment, are provided below. The experiment revealed that LLM-augmented resizing methods led to superior performance in image classification, particularly in cases where maintaining fine image details was critical.

Figure 8. Error and accuracy of a sub-experiment showing the improvements from LLM-augmented methods

Figure 8. Error and accuracy of a sub-experiment showing the improvements from LLM-augmented methods

Illustrative examples of images processed by different methods, including LLM-augmented techniques, are shown below, highlighting the visual differences in the resized images and how LLM-augmented methods contribute to better feature preservation.

Figure 9. Image processed by the Bicubic Method

Figure 9. Image processed by the Bicubic Method

Figure 10. Image processed by the Absolute Energy Method

Figure 10. Image processed by the Absolute Energy Method

Figure 11. Image processed by the Canny Edge Detection Method

Figure 11. Image processed by the Canny Edge Detection Method

Figure 12. Image processed by the Dual Gradient Energy Method

Figure 12. Image processed by the Dual Gradient Energy Method

Figure 13. Image processed by the Hough Transformation Method

Figure 13. Image processed by the Hough Transformation Method

Figure 14. Image processed by the LC Method

Figure 14. Image processed by the LC Method

iv. Conclusion

This experiment highlights the significant improvements brought by LLM-augmented methods in image resizing. By integrating LLM-augmented techniques, LLMSeamCarver can preserve finer image details, resulting in improved performance for image classification tasks. These findings emphasize the importance of selecting the right resizing method in applications where image recognition accuracy is crucial.