Towards Task-Oriented Semantic Communication: A Survey on Technologies, Opportunities, and Challenges

1. Introduction

1.1. Motivation

The forthcoming generation of intelligent communication systems, including 6G and beyond, is anticipated to support intelligent tasks [8,9,10]. Compared to traditional communication, the transformation is largely driven by the proliferation of edge intelligence, autonomous agents, and high-throughput applications such as real-time video analytics, immersive XR, and collaborative robotics [11,12,13]. In such a case, transmitting full raw data is not only inefficient but often unnecessary. Instead, the focus has shifted towards transmitting only task-relevant information that suffices to complete a given goal. Due to this, ToSC has emerged, which transcends Shannon’s classical theory by aligning the communication approach with the specific requirements of a downstream task [1,14]. Through exploiting shared knowledge between sender and receiver, conventional semantic communication systems aim to reconstruct high-level meanings rather than raw signals. However, these systems still often treat semantic fidelity as the final objective. On the other hand, ToSC formulates communication as an end-to-end optimization problem, and the sole objective is to maximize task accuracy (e.g., classification accuracy, trajectory prediction, or object detection precision) under strict constraints on latency, energy, and bandwidth. This distinction underlines a fundamental transition from reconstructing meaning to achieving task objectives through communication. Recent pioneering works have gradually investigated this area. Shi et al. [1] laid a theoretical foundation by modeling ToSC within a task-adaptive information plane and categorizing core technologies, including goal-aligned encoding and joint inference. Zhou et al. [2] extended this view by integrating semantics-aware resource allocation and model adaptation, while Li et al. [3] introduced novel metrics such as task completion ratio (TCR) and goal satisfaction rate (GSR) to replace bit error rate (BER) as evaluation standards. More recently, Zhang et al. [4] emphasized closed-loop task feedback mechanisms, enabling communication policies that adapt in real time to variations in task context, channel conditions, and system load. In spite of increased scrutiny, the field faces several persistent gaps. First, existing surveys either generalize over semantic communication or focus narrowly on physical layer implementations, failing to comprehensively analyze cross-layer ToSC pipelines from sensing to inference. Second, the consideration of ToSC across multiagent, multitask, and cross-device (device-device) communication settings remains underdeveloped, even as distributed edge systems become the norm. Third, prior reviews lack performance summaries or clear taxonomies that would allow fair comparisons across techniques, tasks, or architectural paradigms.

1.2. Comparison and Contribution

Recent pioneer studies such as [15,16] lay the groundwork for the theory of semantic communication (SC), and surveys like [17,18,19,20,21] focus on semantic entropy and information theory, and both tend to lack system-level design principles and alignment with downstream goals. Again, recent works [22,23] in edge AI and federated learning highlight the growing need to fuse SC with the mentioned strategy for resource-efficient inference but provide limited attention to end-to-end optimization, task-specific utility, or multiagent collaboration. The ToSC has received growing attention in recent works. Zhou et al. [2] propose effectiveness-aware metrics and design architectures for task-specific communication; however, they fall short of offering a systematized design flow. Shi et al. [1] combine ToSC with federated learning to reduce latency but lack discussion of deployment scalability and privacy. Li et al. [3] introduce the task-oriented tensor merging of AoI, VoI, and UoI into a unified metric, yet the framework remains theoretically heavy and untested in dynamic networks. Getu et al. [5] attempt to integrate goal-oriented techniques with AI through conceptual frameworks, but practical scalability and security dimensions are underexplored. Zhang et al. [4] introduce the Intellicise network with a feedback-driven architecture and metrics such as semantic interpretability, though it lacks coverage of integration with multiple device-coordination systems. Also, a semantic cross-modal comprehension pipeline powered by AI is presented by Qin et al. [6]; however, cooperative edge settings are discussed in a limited scope. The studies in the aforementioned works primarily focus on the physical layer, overlooking more recent perspectives introduced in ToSC pipelines. In particular, hybrid digital-analog transmission frameworks have been proposed to better align semantic features with wireless infrastructure [24], and generative AI-driven methods enable adaptive encoding prioritizing task-relevant information under channel variations cite Fu2024TasCom, along with privacy-preserving designs to ensure trustworthy semantic processing in multi-task settings [14]. However, concrete architectural frameworks with encoder–decoder modularity remain limited, and systematic approaches for coordinating heterogeneous multi-agent, multitask, and cross-device communication are largely absent. Also, evaluation metrics that reflect practical performance goals are still underdeveloped. More comparison details are summarized in Table 1. To this end, this article provides a summary of core contributions as follows;

1.

The survey revisits task-aligned feature semantics in dynamic channel environments, introducing feature decomposition based on downstream task saliency and analyzing how channel- and task-aware weighting mechanisms shape encoder design, including extensions to hybrid digital–analog semantic pipelines.

2.

A systematic analysis of coordinated inference in multi-agent systems is provided, distinguishing between role-asymmetric coordination (e.g., teacher–student frameworks) and joint decision-making feedback loops, and highlighting the role of adaptive generative semantic encoding for efficient collaboration.

3.

The survey consolidates resource optimization strategies for task-oriented semantic communication, encompassing device–edge co-inference, information bottleneck–based feature compression, and federated edge learning with joint sensing–computation–communication allocation under resource-constrained and non-IID environments.

4.

A future roadmap for task-oriented semantic communication research is proposed, outlining directions such as hybrid-symbolic encoders for compositional generalization, adaptive semantic allocation, and semantics-level privacy preservation in practical deployments.

Table 2. Summary of acronyms used in this article.

Acronym	Description	Acronym	Description	Acronym	Description
AoI	Age of Information	MT	Machine Translation	RTFN	Robust Temporal Feature Network
AI	Artificial Intelligence	MAE	Masked Autoencoder	SC	Semantic Communication
AVP	Audio-Visual Parsing	MOS	Mean Opinion Score	SI	Semantic Information
BPG	Better Portable Graphics	MSE	Mean Squared Error	SDR	Signal Distortion Ratio
CELP	Cell Excited Linear Prediction	MAC	Memory,Attention,and Composition	6G	Six Generation
CNN	Convolutional Neural Networks	MOE	Mixture-of-Expert	SR	Speech Recognition
DJSC	Deep JSCC Coding	NAS	Neural Architecture Search	SE	Squeeze-And-Excitation
DNAS	Differentiable NAS	NTSCC	Nonlinear Transform SCC	SGD	Stochastic Gradient Descent
DOC	Data Oriented Communication	NOMA	Non-Orthogonal Multiple Access	SSMI	Structural Similarity Index Measure
FedAvg	Federated Averaging	OFDM	Orthogonal Frequency Division Multiplexing	ToSC	Task-oriented Semantic Communication
FL	Federated Learning	PSNR	Peak Signal to Noise Ratio	TFN	Temporal Feature Network
GAN	Generative Adversarial Networks	PESQ	Perceptual Evaluation of Speech Quality	UAV	Unmanned Aerial Vehicle
HARQ	Hybrid Automatic Repeat Request	PPM	Prediction by Partial Matching	UoI	Urgency of Information
JSCC	Joint Source Channel Coding	PCM	Pulse Code Modulation	VoI	Value of Information
JSNC	Joint Source Noise Coding	QA	Question-Answering	VAE	Variational Autoencoders
KG	Knowledge Graph	RAN	Radio Access Network	VFE	Variational Feature Encoding
LLMs	Large Language Models	RVs	Random Variables	VIB	Variational Information Bottleneck
LSTM	Long Short-Term Memory	RNN	Recurrent Neural Networks	VQ	Vector Quantization
LDPC	Low Density Parity Check	RL	Reinforcement Learning	VQA	Visual Question Answering
LSTMaN	LSTM-based attention Network	RIB	Robust Information Bottleneck	ViT	Visual Transformers
ML	Machine Learning	RNTS	Robust Neural Temporal Search	WER	Word Error Rate

Table 2. Summary of acronyms used in this article.

Acronym	Description	Acronym	Description	Acronym	Description
AoI	Age of Information	MT	Machine Translation	RTFN	Robust Temporal Feature Network
AI	Artificial Intelligence	MAE	Masked Autoencoder	SC	Semantic Communication
AVP	Audio-Visual Parsing	MOS	Mean Opinion Score	SI	Semantic Information
BPG	Better Portable Graphics	MSE	Mean Squared Error	SDR	Signal Distortion Ratio
CELP	Cell Excited Linear Prediction	MAC	Memory,Attention,and Composition	6G	Six Generation
CNN	Convolutional Neural Networks	MOE	Mixture-of-Expert	SR	Speech Recognition
DJSC	Deep JSCC Coding	NAS	Neural Architecture Search	SE	Squeeze-And-Excitation
DNAS	Differentiable NAS	NTSCC	Nonlinear Transform SCC	SGD	Stochastic Gradient Descent
DOC	Data Oriented Communication	NOMA	Non-Orthogonal Multiple Access	SSMI	Structural Similarity Index Measure
FedAvg	Federated Averaging	OFDM	Orthogonal Frequency Division Multiplexing	ToSC	Task-oriented Semantic Communication
FL	Federated Learning	PSNR	Peak Signal to Noise Ratio	TFN	Temporal Feature Network
GAN	Generative Adversarial Networks	PESQ	Perceptual Evaluation of Speech Quality	UAV	Unmanned Aerial Vehicle
HARQ	Hybrid Automatic Repeat Request	PPM	Prediction by Partial Matching	UoI	Urgency of Information
JSCC	Joint Source Channel Coding	PCM	Pulse Code Modulation	VoI	Value of Information
JSNC	Joint Source Noise Coding	QA	Question-Answering	VAE	Variational Autoencoders
KG	Knowledge Graph	RAN	Radio Access Network	VFE	Variational Feature Encoding
LLMs	Large Language Models	RVs	Random Variables	VIB	Variational Information Bottleneck
LSTM	Long Short-Term Memory	RNN	Recurrent Neural Networks	VQ	Vector Quantization
LDPC	Low Density Parity Check	RL	Reinforcement Learning	VQA	Visual Question Answering
LSTMaN	LSTM-based attention Network	RIB	Robust Information Bottleneck	ViT	Visual Transformers
ML	Machine Learning	RNTS	Robust Neural Temporal Search	WER	Word Error Rate

1.3. Scope of the Survey

As shown in Figure 1, the remainder of this paper is organized as follows. The theoretical foundation, approaches, and communication technologies of ToSC are described in Section II. Section III details the system design, optimization, application, and use cases of Task-oriented Semantic Communication. The future research direction and open issues are presented in Section IV. Section V draws some important conclusions.

2. Theoretical Foundation and Communication Technologies of ToSC

2.1. Theoretic View

2.1.1. Information-Theoretic

The legacy Shannon framework, which primarily ensures the reliable transmission of bits across noisy channels with its emphasis on capacity and distortion metrics such as mean squared error (MSE), has underpinned decades of progress [25,26]. It is becoming less effective for modern applications in which communication is required to support higher-level inference and decision-making. What matters is not whether a signal is perfectly reconstructed but whether the transmitted information is sufficient for the intended task. Recent studies have extended the use of information-theoretic principles to account for semantic entropy, task relevance, and goal-oriented rate–distortion tradeoffs [27]. To demonstrate the ineffectiveness of traditional rate-distortion formulations, Kountouris et al.   [28] introduced the concept of semantic entropy, which quantifies uncertainty in task-relevant semantics instead of in raw data. The concept of task-oriented rate-distortion was then formalized in later research [29,30], in which encoders and decoders are optimized for performance in some tasks like classification and object recognition as well as reconstruction fidelity, taking into account practical constraints like bandwidth and latency. To analyze ToSC, the information bottleneck (IB) principle [31] creates a foundation for framing the design objective as a balance between compression and task relevance. Due to its variational adaptation named Variational IB (VIB) [32,33,34], which guarantees that the encoded features retain minimal and sufficient information about the target task and discard task-irrelevant details, it has proven very helpful for wireless SC. This is highly relevant to wireless settings in which devices need to adjust transmission rates to channel conditions without sacrificing semantic relevance. Studies like Hu et al. [35] focused on integrating information-theoretic concepts at the system level. Specifically, they investigate multi-user SC systems based on DL for transmitting both single and multimodal data. Further studies on AI-driven SC frameworks by Qin et al. [6] highlighted the practical application of representation learning to establish information-theoretic objectives. Zhang et al. [4] extended these ideas by introducing interpretability and task alignment as critical design dimensions, thus broadening the scope of the traditional rate–distortion problem. To guarantee task effectiveness under practical constraints, the aforementioned viewpoint naturally encourages rethinking of wireless system design to embed semantic principles into resource allocation, coding techniques, and network architecture designs.

2.1.2. Implications for Integrated Wireless System Design

The theoretical advances of ToSC set a foundation for rethinking wireless system design, particularly in how encoding, decoding, and resource scheduling are orchestrated. The encoding schemes under ToSC actively suppress task-irrelevant information, transmitting only those features that are indispensable for downstream inference [36]. Such representations not only reduce bandwidth consumption but also mitigate the computational cost at the receiver because lightweight NN models can operate directly on compressed latent features instead of fully reconstructed signals [29,33]. A key implication for wireless systems is the requirement for an adaptive strategy to align semantic encoding with varying channel conditions. Instead of treating all bits with equal importance, it emphasizes the preservation of task-relevant features to ensure performance degradation under adverse conditions remains bounded  [18,37]. To guarantee robust task performance with varying link quality, it is necessary to incorporate channel state information (CSI) into the encoding process so that task-relevant bits are prioritized in transmission  [6,35]. It is important to note that by aligning resource allocation policies with semantic importance, a system can achieve a more effective balance between communication efficiency and task reliability. Figure 2 summarizes the theoretical point of view of wireless communication encoders’ evolution, specifically from classical encoders to semantic encoders.

2.2. Approaches to ToSC

Any engineering problem must be fully understood, including any assumptions, limitations, and oversimplifications that might need to be resolved. The overview of approaches to task-relevant feature representation, illustrating architectural strategies for structuring, weighting, and encoding semantics to optimize communication performance, is summarized in Figure 3. Therefore, we use this part to provide an overview of the various theoretical and practical approaches proposed for developing ToSC frameworks.

2.2.1. Probabilistic

Initially, Rudolf and Bar-Hillel introduced a foundational theory of SI [38] that aimed to move beyond the statistical framework of Shannon’s information theory and incorporate logical probabilities to define the informativeness of statements. To move the emphasis from syntax-based information measurements to a probabilistic approach based on modal logic, Rudolf work suggested that the logical truth of a statement should be used as the primary measure of information content. The basis for a task-oriented interpretation of semantic communication was established by this framework, which treats all communicated data as equally important but prioritizes information according to its relevance to a particular task. The probabilistic approach identifies and prioritizes task-relevant SI by modeling the uncertainty and variability of source signals. Modern ToSC systems reduce transmission cost and retain important information with the help of Bayesian inference and probabilistic reasoning to modify transmission strategy in response to channel variations. For example, very recently, probabilistic semantic encoding has been studied in wireless communication networks to optimize data transmission efficiency, mitigating the impact of channel noise and network congestion [39].

2.2.2. Hierarchical Graph

The most common way to define semantics in the literature is by using a structured knowledge graph (KG). However, numerous terms, including ontology, taxonomy, and semantic network, are applicable to represent the same meaning [40]. We consider these approaches KG-based SC because they are all essentially the same (i.e., using a graph structure, or KG, to represent knowledge in the system. Semantic parsing in task-oriented conversations is a prominent use case for hierarchical graph representations. To describe compositional inquiries, Gupta et al. [41] presented a hierarchical annotation framework. In particular, the nested and interconnected nature of user requests is captured by this system, which uses a tree structure with nodes representing intents and slots. Such a representation enables standard constituency parsing models to efficiently and accurately interpret complex queries, outperforming traditional sequence-to-sequence approaches. In addition, hierarchical graph-based methods have been applied to semantic communications with adaptive channel coding. For instance, a task-oriented scene graph-based semantic communication system named GRACE has been proposed [42]. GRACE utilizes scene graphs to represent the semantic content of visual data hierarchically, enabling adaptive coding strategies that improve communication efficiency and robustness. This approach is popular because it is simple to define semantic similarity metrics by defining knowledge over a graph and analyzing it using sophisticated graph theory techniques. The possibility of using recent learning techniques across such graphs has been made possible by current studies on graph neural networks.

2.2.3. Significance-Based Data Prioritization

An alternative perspective on semantics emphasizes significance-based data prioritization, which we denote as significance-based semantic communication (Sc). When compared to the previously discussed definitions, this approach centers on the principle of prioritizing information according to its practical relevance instead of its linguistic or representational meaning. It is categorized as a task-oriented approach with the primary aim of addressing the effectiveness problem, the third level in the hierarchy of communication problems. Instead of focusing on conveying the complete meaning of information, significance-based Sc concentrates on ensuring the transmission of the right information. Here, what qualifies as right is not absolute but determined by the application context and the intended goal, thereby framing communication as a task-oriented process. The ultimate value of data lies in its contribution to achieving the desired outcome. Recent works have explored strategies that allocate transmission resources to the most task-relevant information. For example, Wang et al. [43] propose an importance-aware JSCC framework that selectively transmits task-important features, quantified by a new metric named task-oriented semantic spectral efficiency and optimized via RL. Sun et al. [44] introduce a scheme that adapts transmission rate based on both channel conditions and semantic importance, using a contribution-based importance analyzer and a semantic transmission integrity index to preserve task performance while optimizing resource use. Also, Ma et al. [45] explore multimodal systems in which visual data are segmented into task-relevant semantic units, prioritized at the segment, token, and bit levels to maximize efficiency under noisy wireless channels.

2.2.4. Machine Learning

Utilizing ML techniques to "learn" the interpretation of the engineering problem is another popular approach that is experiencing a spike in interest. Like the debate over model-based versus conventional approaches to general inference problems, an established KG drives model-based semantics in solving engineering problems. In contrast, machine learning techniques leverage the power of feature learning from source data to develop these semantics [46]. Employing methods derived from CV and NLP, deep networks have been trained to communicate optimally while maintaining semantic content [47]. Similarly, these networks have shown improvement over time and even adjusted to dynamic changes in the communication problem by applying reinforcement learning (RL) methods. We term this strategy an ML-based approach. To simplify, this section offers a high-level summary of these approaches and the multiple ways each tries to represent task-relevant information from the source data. Approaches to ToSC have been divided approximately into four categories based on these semantic definitions. Reflecting on factors beyond the statistical modeling of the symbols used to represent the information, we broadly define semantics as any definition of information or its transmission based on previous and recent literature. Though something beyond definition encompasses diversity, this point of view seeks to bring together the definitions mentioned earlier. They do not, of course, conflict with one another.

2.3. Feature Extraction and Representation Learning

In Section 1, we introduced traditional communication models that often focus on complicated compression of raw data for transmission, in contrast to ToSC models that shift toward understanding and transmitting the underlying semantics of the explicit task. A key challenge is representing the most relevant task features, including structured and unstructured data types, such as images, text, or sensor inputs, while minimizing communication overhead.

To address this challenge, recent advancements in ML and DL have opened up exciting possibilities for representing traditional transceivers or individual signal-processing components using DNNs[47,48,49]. Drawing inspiration from autoencoders in DL, an E2E communication system has been introduced, aiming to unify the signal processing blocks traditionally found in communication systems [50]. In particular, feature selection and feature extraction techniques taken from DL are used to discover task-relevant characteristics while minimizing any trivial information. The system gives priority to features that optimize task performance by integrating task-specific knowledge into the communication model to improve its efficiency.

2.3.1. Textual Data

Inspired by the success of DL in NLP applications such as machine translation [51,52], the authors of [53] pioneered the use of DL for text transmission. The system model proposed in [53] uses global vectors for word representation, first representing words as embedding vectors at the transmitter. The embedding vector is then mapped into a finite number of bits to communicate sentences to a receiver using an LSTM-based encoder and decoder, geared by success in machine translation sequence-to-sequence learning frameworks. The extracted semantic features are then used in the sentence recovery process at the receiver. LSTM-based feature representation delivers the lowest WER in a low SNR regime for a finite coding length compared to Huffman and Gzip. Also, with the increase in sentence length and under a specific SNR range, LSTM-based feature representation achieves superior performance because of the relevance of extracted information. Nevertheless, LSTM-based feature representation has difficulty scaling efficiently with long sequences. The common word representation models, specifically global vectors and word vectors, only capture word relationships without being able to express syntactic characteristics [54]. The vanishing gradient problem limits learning from distant past inputs and makes it impossible to parallelize across time steps. Consequently, it is challenging to handle complicated sentences using this strategy since it only explains the likelihood that a particular word would appear after another in a sentence.

Given the challenge mentioned above, recent Transformer models have garnered considerable interest since it successfully extracts task semantic features and syntax from entire sentences[47,49]. Combining the Transformer network with multi-head self-attention mechanisms enables it to extract and represent different attributes from input texts simultaneously. Consequently, the Transformer network learns long-range correlations in parallel input sentences with less computational cost than a sequential input-based LSTM. Also, the standard Transformer processes a sequence in parallel, applying fixed layers of self-attention without the ability to refine its representations over time. Effective representation of long-range dependencies in the input sentence may be limited by the multi-head attention’s inability to grow across layers to fine-tune token relationships.

To overcome the aforementioned SC challenge, Zhou et al. [56] apply iterative self-attention leveraging the power of a Universal Transformer (UT) [57]. Each token goes through a number of changes within the same layer, improving its representation. To better capture intricate dependencies and nuanced relationships, UT employs a recurrent approach to enable tokens to iteratively change their representations compared to applying a fixed number of layers. Adaptive Computation Time (ACT) controls the iterative refinement mode, making it more flexible and dynamic and allowing the model to adjust and improve its feature extraction and representation capabilities during processing. This helps extract richer, more nuanced text representation. Incorporating an adaptive circulation mechanism into the standard SC system enhances its flexibility, allowing it to better handle varying physical channels with different SNRs. This adaptability feature enables the system to adjust transmissions according to different SNR conditions, improving overall communication performance. When tested under a high-SNR regime with a noise level of about 10 dB, UT performs better despite the noise. However, it fails to transmit the sentence correctly in low SNR regions.

However, standard Transformers and UT face high communication overhead, slow convergence, and heavy memory usage in distributed communication systems due to their large size, leading to inefficiently distributed training. The increased latency and high resource intensity make them less scalable. To address the face challenges, model pruning effectively minimizes communication overhead by eliminating redundant parameters, thus reducing the volume of data exchanged during distributed training [55]. Also, pruning simplifies the model structure, accelerating convergence by enabling faster updates and requiring fewer training iterations. Also, network quantization has been found to alleviate memory constraints by representing weights and activations in lower-precision formats, significantly reducing storage and computation requirements [55]. As shown in Table 3, L-DeepSc [55] has fewer parameters compared to the non-compressed model [47] while achieving the same BLEU score. This result underscores the critical role of ToSC in improving communication efficiency by minimizing data transmission demands through effective representation and extraction of task-specific features.

2.3.2. Visual Data

In existing visual data communication systems, pixel intensities serve as the representation unit, ensuring the reconstructed visual output at the receiver looks exactly like the transmitted one. Conversely, in ToSC, feature representations and extraction tolerate some pixel defects since they extract the image’s underlying semantics at the expense of pixel fidelity. In both systems, the output must achieve the downstream task with minimal allowable distortion and high accuracy for visual data reconstruction and inference tasks.

To accomplish feature representation and extraction before transmission, the initial few layers of a CNN-based architecture act as a feature extractor [58,59,60]. For feature extraction from a source image, ResNet101 and the generalized-mean (GeM) pooling layer, followed by the normalization layer, are employed. The semantically guided code head and PQ code were then applied before quantization. This enables the wireless channel to send the query image with strong retrieval performance and a constrained bandwidth. Compared to Resnet101, a Resnet152 model that acts as a multi-level feature extractor has shown remarkable achievement in extracting spatial semantic features and segmentation semantics from an input image [58]. On the other hand, Huang et al. [59] employ a convolutional semantic encoder to derive the semantic features of image data, capturing information such as category, spatial arrangement, and visual features as the fundamental representation units. The convolutional feature extraction used by [59] has been modified with the feature attention mechanism to optimize transmission rate by regulating the source compression ratio and channel coding rate based on channel state information [60]. For delay-sensitive tasks, particularly in cooperative edge inference systems, deploying the first few layers of VGG proves to be an effective method to extract task-relevant local features [61]. In Figure 4, features $Z_1..Z_k$ are extracted from the output of multiple camera views. In particular, this approach adheres to the IB principle [31], which has been demonstrated to work well for extraction in edge inference systems with multiple devices. The goal is to minimize the mutual information between the extracted feature Z and the input image X. Simultaneously, maximize the mutual information between the encoded feature and the target variable. The extracted features are then encoded into a latent feature space, exploiting the mean and variance of distributed feature vectors. One way to think of the extracted features is as the convolution kernel’s signal components. Different scaling factors, known as channel-wise soft attention, are created for various channel features to either strengthen or weaken their connection to the following layer based on the channel relationship that has been recorded. Learning task representations, both models perform better than the traditional JPEG-LDPC under Rayleigh, AWGN, and Rician channels, enabling larger compression ratios without compromising perceptual quality. Due to their localized receptive fields, traditional CNNs such as ResNet and VGG exhibit limitations when handling global dependencies in visual data [62]. The drawback leads to suboptimal performance in tasks requiring a global image understanding, including semantic segmentation. To mitigate, Vision Transformers (ViT) [63] have been deployed as an alternative. ViT uses self-attention to capture global dependencies, treating images as sequences of patches. In particular, the pre-trained masked autoencoder combined with ViT has successfully been employed to extract and transmit image features [64]. The encoder processes the original image using a sequence of Transformer blocks after removing the masked patches and embedding the unmasked patches with their positional information. Training time and memory usage are significantly decreased because the encoder only needs to process a tiny percentage of unmasked patches and map them to the encoded features for transmission [64]. These features are then encoded with a learnable shared codebook to combat semantic noise and achieve a 0.95% compression ratio compared to the conventional JPEG-LDPC. The model maintains consistent classification accuracy when trained under the AWGN channel with an SNR of 0 dB and tested across AWGN, Rayleigh, and Rician channels with SNRs ranging from -6 dB to 18 dB under semantic noise conditions. This significantly outperforms conventional methods, particularly at low SNR levels [64].

While standard ViT offers superior performance for tasks requiring global feature interactions, its reliance on large-scale datasets for training and computationally intensive operations makes it less practical for many real-time and resource-constrained applications. Liu et al. [65] introduce SwinJSCC, which constructs hierarchical feature maps in the latent space while maintaining linear computational complexity relative to the image size. The SwinJSCC framework built over a Swin Transformer [66] addresses the challenges as mentioned earlier by introducing a sliding window mechanism for self-attention, which balances local and global feature extraction while significantly reducing computational overhead. SwinJSCC uses feed-forward networks and the multi-head self-attention module to process image patches. The model extracts long-range dependencies in the image thanks to the shifting window-based self-attention method. It applies self-attention inside each window after dividing the image into a grid of windows. Table 4 shows that SwinJSCC outperforms both traditional compression techniques, BGP-LPDC and Attention Deep JSCC [60], specifically on BD-rate, inference time, and encoding latency. Despite these improvements, SwinJSCC remains resource-intensive in feature extraction for edge and low-power device deployment. Lightweight frameworks, including MobileViT [67], adapt attention-like mechanisms for edge computing and may be explored to enhance efficiency further. Its compact architectures, pruning, and quantization capabilities significantly reduce resource consumption while maintaining inference accuracy.

2.3.3. Audio Data

Modern speech representation and extraction methods, as summarized in Table 5, have recently shifted to ML models, particularly DNNs. The primary driving force has been the capacity of DNNs to capture essential patterns through feed-forward and back-propagation by exploiting stochastic gradients and substantial training data adopted from CV techniques. In traditional speech communication systems, waveform amplitudes are the primary representation units, ensuring that the reconstructed audio at the receiver closely mirrors the original transmitted signal. On the contrary, ToSC emphasizes transmitting the underlying semantic content of audio, allowing for minor waveform distortions. This approach is particularly beneficial for speech recognition tasks [68], and the focus is on the semantics conveyed rather than speech reconstruction. While the majority of outputs of DNNs’ intermediate nodes’ values range between 0 and 1, final output layers use categorical softmax for speech synthesis or the standard softmax to output a range of probabilities. Such models represent high-dimensional, multi-class input speech signals in a low-dimensional latent space. The compressed latent space is then used in tasks such as Automatic Speech Recognition (ASR) to predict posterior probabilities of classes present in the input speech signal. Deep learning models, especially CNNs, are employed to extract and represent semantic features before transmission. The WaveNet architecture utilizes dilated convolutions to capture long-range temporal dependencies in audio signals, effectively increasing the receptive field without significantly increasing computational cost [69]. A DL network is effectively sparsified by systematic skipping, as necessary information in speech is frequently dispersed widely across subsequent input samples. WaveNet, which is significantly more expensive, has demonstrated superior speech quality compared to more conventional vocoders, employing a quantized feature representation [70].

However, the localized receptive fields of conventional CNN-based models such as those modeled after WaveNet can make it difficult to capture global dependencies [71]. Performance on tasks requiring an efficient understanding of the audio background is limited. To address this, transformers designed specifically for audio data, like AST, have been deployed [72]. AST is an attention-based model that captures long-range dependencies in audio signals, achieving SOTA results on various audio classification benchmarks. AST has been introduced to handle audio classification tasks by learning direct mappings from audio spectrograms to corresponding labels. The model converts audio signals into spectrograms to extract robust semantic features. The extracted features are then encoded and quantized for efficient transmission. Transformer-based models present difficulties in resource-constrained applications as they rely on massive datasets and have high computational demands in processes [73]. To address this, lightweight frameworks, including MobileViT [67], are used to modify attention processes in cases including edge computing. To cut down on resource usage without compromising task accuracy, models use pruning, quantization, and compact architectures [74]. It is important to note that lightweight models may trade off some degree of accuracy for efficiency and impact performance on complex and high-resolution tasks.

2.4. Semantic Encoding and Compression

As shown in Figure 5, the Shannon separation theorem requires the source and channel coding in classical communication systems to be designed as separate modules [25]. Leveraging the statistical characteristics of the input data, source coding tends to minimize redundancy, and channel coding guarantees robustness to transmission noise and interference. For instance, traditional compression standards like JPEG, H.264/AVC, and H.265/HEVC for lossy multimedia data [75] and Huffman and arithmetic coding for lossless instances [76,77] function at the syntactic level.

Although the mentioned techniques are good at reducing bit rates, they are limited in distinguishing between perceptual or statistical significance information and task-related relevant information. Incorporating a relevance view of information semantic communication steps beyond the syntactic approach. In other words, semantic encoding aims to capture and retain features of the source signal that are directly important to the inference task rather than accurate lossless reconstruction [17,78,79]. The optimization objective takes into account not only the bit rate and distortion but also the semantic fidelity of the transmitted representation, reflecting a larger rate–distortion semantics trade-off [80,81]. Semantic encoders in practical applications are frequently implemented with DL architectures that can extract high-level features from raw input data. For example, in downstream tasks, convolutional neural networks and Transformer-based models have demonstrated efficacy in extracting features from inputs into compact and sufficiently informative embeddings that are robust to channel impairments [20,82,83]. The compact embeddings can be further compressed using learned quantization [84] or low-rank feature factorization [85] before transmission.

Compared to conventional codecs, which are designed for human perception or generic data fidelity, semantic encoders are jointly trained with task decoders, often in an end-to-end fashion, with the channel model embedded in the training loop to directly optimize task accuracy under bandwidth and noise constraints. Compression, for example, in semantic image transmission for object detection, background pixels that do not contribute to bounding box accuracy can be heavily compressed or discarded, whereas salient object features are preserved with higher precision [86]. In wireless sensing applications, semantic encoders selectively transmit features related to environmental cues relevant to decision-making [19]. This selective preservation enables substantial savings in bandwidth and energy while maintaining or even improving task performance compared to conventional bit stream transmission. Since the semantic encoding is advancing, the conventional separation between source and channel coding becomes less effective in digital communication. Feature extraction, compression, and channel adaptation must all be taken into account in the optimal design for low-latency communications as the separation theorem’s asymptotic guarantees are not achievable [80,86].

To co-optimize semantic encoding and channel robustness in a single design the Joint Source–Channel Coding (JSCC) architectures are used for this purpose. For more discussion, see SubSection 2.5.

2.5. Joint Source-Channel Coding (JSCC)

Reliable transmission has long been at the heart of wireless communication, and the transition toward ToSC redefines how channel coding should be designed and applied. Traditional error control schemes, such as turbo codes and LDPC codes, are optimized to minimize bit or block error rates under the Shannon framework. However the primary objective is not necessarily accurate bit recovery, but preserving semantic integrity and ensure received information suffices for the downstream task. This demand has motivated a new generation of coding strategies to jointly consider semantic relevance and robustness under varying channel conditions. Recent works have begun to incorporate task-aware design into channel coding frameworks. For example,  [86,87] have gained significant traction in SC, as they learn end-to-end mappings that inherently adapt representation to task requirements without rigid separation between source and channel coding. By leveraging DNNs, these schemes enable acceptable distortion under poor channel conditions, allowing task performance to remain acceptable even when bit-level accuracy deteriorates. Extensions of JSCC to multi-modal and multi-task settings further highlight its adaptability to diverse task-oriented environments [33,34].

The Figure 5 presents a direct mapping of extracted source signals to channel inputs, aiming to minimize transmission volume. Jointly handling source variations and channel impairments JSCC reduces both error propagation, computational overhead and bandwidth usage. All SI are encoded in a format that can be effectively interpreted by the decoder, enhancing robustness under noisy conditions [88]. Recent studies focus on advancing JSCC through the use of DNN [89,90] to improve the semantic extraction, processing, and transmission of rich media data such as images and videos. The targets is to improve beyond conventional convolutional based designs by addressing challenges related to computational efficiency, training scalability, and cross domain generalization. Also, developing modular JSCC architectures compatible with existing communication protocols and deployment requirements is a primary objective.

In addition, the integration of transformer-based backbones is explored to enhance data transmission performance, and topological deep networks are also investigated for semantically aligned encoding. The adaptive JSCC techniques are also proposed to accommodate varying signal structures and application contexts for future mobile communication systems to enable real-time adaptation to channel conditions, device constraints, and task requirements  [91,92]. Robustness in TOSC also relies on adaptive coding and modulation strategies to dynamically allocate resources based on semantic importance. Instead of uniform protection of all transmitted symbols, adaptive schemes aim at prioritizing features specific to inference, leading to an effective integration of unequal error protection (UEP) into semantic frameworks [93]. Another means is the integration of semantic feedback and RL into robust transmission through incorporating task-aware feedback loops; thus, systems can adjust coding rate in real time to maintain task accuracy under dynamic network conditions [6]. While early DeepJSCC frameworks learn fixed end-to-end mappings under predefined channel distributions [86,87], practical ToSC systems require dynamic adaptation across varying channel conditions, heterogeneous task priorities, and device constraints. Joint source–channel adaptation therefore extends beyond static JSCC by enabling semantic encoders to adjust compression ratios, latent dimensionality, and transmission strategies based on instantaneous CSI. Adaptive rate control mechanisms allow semantic encoders to produce scalable embeddings whose dimensionality varies with SNR or latency budgets, improving robustness under non-stationary wireless environments [33,91]. Instead of transmitting fixed-length feature vectors, adaptive JSCC schemes learn to allocate bandwidth proportional to semantic importance and channel quality.

2.6. Modulation and Resource Allocation

Traditional wireless systems optimize modulation schemes and resource allocation primarily for throughput, spectral efficiency, and bit error rate. However, the primary goal of ToSC is not uniform data reliability but the preservation of task semantics under dynamic and resource-limited environments. This calls for semantic adaptation across both waveform design and resource distribution. In the evolving landscape of ToSC, modulation and resource allocation transcend traditional metrics like spectral efficiency and bit error rates, turning instead toward the retention of task semantics. Conventional modulation schemes largely ignore data relevance, though semantic-oriented systems adapt symbol mapping and constellation design to reinforce features vital for downstream inference. Adaptive schemes often couple semantic importance with physical layer decisions, allowing robust protection of relevance features and discarding irrelevant information [98].

To maximize AI task performance under resource restrictions, Liu et al. [99] developed a joint bandwidth and power allocation framework to dynamically balance communication resources and compression rate. To boost long-term user satisfaction in energy-harvesting cognitive radio NOMA systems, Yan et al. [100] then further introduced a QoE-aware semantic allocation technique to concurrently adjust transmit power, time-slot division, and semantic compression. In addition, beamforming, subchannels, and bandwidth are allotted by RL agents in adaptive resource schemes based on semantic-bit quantization to optimize the overall quality-of-service (SC-QoS) of semantic communication. To control resource sharing between Wi-Fi and vehicle users inside unlicensed bands, Shao et al. [101] introduced metrics including high-speed semantic transmission rate and semantic spectral efficiency in their deep RL scheme for 5G V2X heterogeneous networks.

2.7. End-to-End Optimization

Treating the entire communication pipeline from feature extraction to task execution as a single differentiable framework, the ToSC system is optimized holistically using ML models, enabling all system components to undergo joint design and training to achieve superior task-specific performance. This approach focuses on minimizing task loss functions, such as cross-entropy for classification, rather than conventional communication metrics like bit error rate. Also, facilitating SI extraction by training the semantic encoder to focus on task-relevant features reduces transmission cost. Differentiable channel models help establish robustness under noisy and constrained channels, and compression attending to salient SI [102]. Based on this, in settings with limited bandwidth, task-oriented systems perform more effectively and efficiently than conventional communication.

2.7.1. Semantic Noise

The physical noise sources, such as quantization errors and channel distortions, affect signal quality at the physical layer. Besides, semantic noise arises due to distortions in the understanding of transmitted information. These distortions may result from ambiguity in representation, contextual misalignment between the transmitter and receiver, or errors introduced by model generalization [109]. In models like DeepJSCC, E2E neural networks are employed for joint source and channel coding, and mitigating semantic noise becomes crucial to preserving the fidelity of transmitted information. Semantic noise was observed to seriously impair communication systems’ efficiency, leading to a mismatch between the transmitted and received semantic symbols [109]. Two distinct kinds of semantic noise are identified, namely adversarial semantic noise and literal semantic noise. Adversarial semantic noise occurs when attacks are induced to the embedding layer via the semantic channel. In contrast, the latter is caused by textual errors [64]. In particular, there are differences in the causes that lead to semantic noise for various sources, such as audio, images, and text. Textual semantic ambiguity, whereby minor words change, such as the substitution of synonyms, may cause the DL model to interpret a sentence’s semantic meaning incorrectly[64].

To successfully mitigate the effects of semantic noise with less transmission cost, it is crucial to predict the noise in semantics within the communication systems before the ToSC system design. A robust ToSC framework against semantic noise was proposed in [64]. The work suggested that adversarial training be used to include the samples containing the training dataset with semantic noise to eliminate the impact of semantic noise, with part of the input masked using the masked autoencoder (MAE). To enhance the resilience of ToSC systems, the vector quantization-VAE (VQ-VAE) is integrated into a shared discrete codebook to represent encoded features between the sender and receiver. In [49], a robust deep learning-enabled ToSC system named R-DeepSC is presented that deploys adversarial training and a calibrated self-attention mechanism to address semantic noise. To enhance semantic fidelity in a range of communication settings, adaptive encoding-decoding techniques are incorporated [110]. Also, knowledge graphs are incorporated to improve the transmitter and receiver’s shared contextual understanding and reduce misunderstandings brought on by imprecise representations[111].

2.7.2. Training Objective and Metrics

Traditional communication systems often employ ordinary loss functions for data fidelity. A task-specific loss function tailored for optimization aligns the transmission objectives with specific tasks. To optimize source representation efficiency and task accuracy, integrate components like task loss to maximize downstream application performance, latent space regularization to promote compact and efficient representations, and reconstruction loss to ensure data recovery quality. For example, image reconstruction tasks could rely on Mean Squared Error (MSE), unlike classification tasks, which could use cross-entropy loss or its derivatives. Future frameworks should introduce channel-aware loss functions to account for the impact of noisy or unreliable communication channels, ensuring robustness. However, this article introduced channel distortion and regularization terms to penalize the mismatch between the transmitted vector and the noisy received vector during model training, improving model robustness. Table 6 highlights the diversity in loss functions, catering from latent Regularization and channel robustness to downstream task efficiency, balancing competing priorities such as task accuracy and communication efficiency. Designing a practical loss function is critical for training encoder-decoder models in ToSC systems. The loss function must balance task-specific accuracy, semantic fidelity, communication efficiency, and robustness to channel noise. This is achieved by combining task-oriented losses such as cross-entropy or mean squared error with additional terms for semantic alignment, rate-distortion trade-off, and noise resilience. Multitask loss formulation model parameters tuned to prioritize explicit objectives, including minimizing communication costs while preserving task-relevant features. Incorporating techniques like perceptual loss, cosine similarity, or reinforcement-based feedback further enhances communication performance. The challenge lies in achieving an optimal balance between learning feature representation and cost, ensuring generalization and robustness under diverse communication conditions.

Evaluating system performance in ToSC requires a range of metrics that measure both the quality and accuracy of transmitted information or inference tasks. Standard metrics, from the F1 score to assess classification tasks to word error rate (WER), are critical for evaluating speech data’s accuracy. The perceived quality of reconstructed speech or audio is also assessed using the mean opinion score (MOS). Metrics such as the Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) evaluate the visual integrity of transmitted pictures for image-based tasks. Also, the mean opinion score (MOS) can be used to measure the perceived quality of reconstructed speech or audio. For image-based tasks, metrics like the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) assess the visual fidelity of transmitted images [112]. More metrics are summarized in the Table 7.

2.7.3. Differential Design for Loss Minimization

To optimize loss functions in ToSC, a differential design technique is needed to modify loss components according to semantic significance, task importance, and channel conditions. Adaptation using gradient-based optimization and reinforcement learning feedback[113] has been adapted. As a result, the system can reduce inference errors and still ensure effective transmission and be robust to both physical and semantic noise. Integrating primary task-oriented losses, such as cross-entropy, with adaptive constraints like rate-distortion optimization and contrastive learning to enhance semantic alignment. Recent studies suggest that adjusting loss weights using adversarial training improves convergence and generalization across diverse communication environments [114]. However, achieving an optimal balance remains challenging due to the complexity of multi-objective tuning and computational efficiency constraints. By enabling the E2E optimization of all system components through unified gradient-based learning. It is possible to integrate the semantic encoder, channel models, and task decoders into a single trainable framework, ensuring a seamless flow of task-relevant information. For example, Authors in [47] pioneered the design of a model that utilizes Transformer architectures to prioritize semantic accuracy over conventional metrics, significantly improving communication efficiency under noisy channel conditions. In particular, this approach ensures inference optimization and fosters adaptability and robustness in communication systems.

For ToSC involving structured prediction problems, loss functions are developed to incorporate symbolic knowledge into DNN training. This ensures the model is trained to prioritize transmitting specific features and that the outputs adhere to the desired structural constraints [115]. In the image transmission regime, ToSC systems have employed rate-distortion theory to balance the trade-off between compression efficiency and task performance, optimizing the system for specific applications such as image classification [116]. Moreover, unified semantic communication systems have been proposed in multi-task scenarios, and domain adaptation techniques are utilized to design loss functions that cater to multiple tasks simultaneously, thus reducing transmission overhead and improving generalization across tasks. By customizing the loss function, it is feasible to enhance the performance and efficiency of ToSC systems.

2.7.4. Error and Noise Robustness

E2E optimization enables the entire communication pipeline, from semantic encoding, channel modeling, and decoding, to be jointly trained using specific channel-aware loss functions that inherently account for channel impairments. In particular, Xie et al.[47] demonstrated how integrating noise-aware loss functions into the DeepSC framework improved semantic robustness by enabling the system to encode specific task features. Similarly, Bourtsoulatze et al. [89] proposed semantic-aware transmission strategies that use DNNs to encode features resilient to channel noise, significantly improving model accuracy. Denoising autoencoders and attention mechanisms further enhance error robustness by emphasizing the reconstruction of essential semantic content even under severe distortion. Moreover, let’s think of adversarial training to simulate worst-case channel conditions during training, ensuring the system remains robust against unexpected noise scenarios. By jointly optimizing all components, ToSC systems tolerate noise and excel in resource-constrained environments, prioritizing communication task performance over traditional metrics such as bit error rate. These approaches highlight the transformative potential of E2E optimization in enabling resilient and efficient task communication.

2.8. Data Reconstruction and Inference Tasks

2.8.1. Reconstruction Tasks

In practice, source data transmission, such as image, text, speech, video, or multimodal, incorporates a complex process of source compression (source coding), error correction (channel coding), and adaptive transmission techniques to guarantee efficiency, accuracy, and minimal latency. Unlike conventional communication, ToSC aims to successfully transmit the SI from the source instead of accurately receiving each bit or symbol[117]. The transmitter must learn, extract, and communicate only global semantic features to enable the receiver to recover both small- and large-scale source data features. The existing digital communication systems require SI to be mapped into compatible bits before transmission using binarization [53] and vector quantization techniques [118].

These features are mapped into indices using vector quantization (VQ) [119], converted into bits, and sent via existing digital communication systems. A robust VQ-SC system was proposed in [120] to minimize semantic noise, and it performs better than conventional techniques. The robustness of a VQ-VAE system against noisy wireless channels was demonstrated in [121], which exhibits potential as a DL representation of complex and diverse data distributions, hence positioning it as a viable solution for a wide range of wireless communications applications, including data recovery. Further, Fu et al. [118] designed a DL-enabled VQ that learns and extracts multi-scale SI and maps it into complex channel symbols through multi-scale space embedding features. Therefore, this section discusses image, text, and speech transmission in detail to demonstrate the data recovery task.

Text reconstruction: While traditional communication systems concentrate on optimizing individual physical layer blocks, combined transceiver optimization is now possible with DL-enabled E2E systems. Several authors are trying to improve communication systems from a semantic perspective by designing sophisticated DL-enabled E2E communication systems that incorporate all of the blocks of conventional communication systems’ physical layer, enabling collaborative optimization of transceivers. Fueled by DL, NLP has made significant progress in understanding and analyzing many words or texts. In particular, Xie et al. [47] propose groundbreaking work on DL-enabled ToSC to facilitate text recovery: a unique method for communication systems, DeepSC, powered by the Transformer architecture to learn textual semantic features and reconstruct the exact text content sent by the transmitter at the receiver. DeepSC focuses on recovering sentence meaning instead of typical bit- or symbol errors to maximize system capacity while minimizing semantic errors. Adopting transfer learning strategies [47] speeds up model training and guarantees DeepSC’s applicability in various communication contexts. This method also proposed phrase similarity, a novel metric for precisely evaluating Scs performance. A DeepSC (R-DeepSC) was proposed in [49] to minimize semantic noise and facilitate transmission reliability. Also, the author in [56] improves the handling flexibility of various semantic information inside the text by including an adaptive circulation mechanism and leveraging the power of a Universal Transformer. This improvement overcomes a restriction of fixed transformers used in [47] by enabling more efficient transmission of text with different semantic meanings. Moreover, [55] extends the application of DeepSC introduced in [47] by designing a lightweight DL-SC system named L-DeepSC that leverages the power of edge devices’ computing capabilities to transmit and recover text data. For successful text transmission via the ToSC system, Figure 6 summarizes the text transmission unified DL system framework, showing the source data feature learning task at the transmitter and the text recovery task at the receiver.

Image reconstruction: Recent large volumes of image data in communication introduce transmission issues such as latency and network congestion, which are challenging for conventional communication due to limited capacity. It is feasible to significantly minimize the quantity of data that needs to be sent over the wireless network while maintaining the semantic fidelity of the image by using image ToSC. Data compression with DNNs has advanced recently, especially for image compression [122,123,124], unlike the majority of conventional image compression standards, such as JPEG, JPEG2000, and BPG, which rely on transform coding, quantization, as well as entropy. Prior works at semantic image communications have successfully explored autoencoder-based DNNs[58,60,87,89], to develop the JSCC approach for image transmission and showed good performance in the limited bandwidth and low SNR regimes.

At all SNR values, the study [89] achieves higher SNR than existing separation-based conventional communication by learning noise-resistant coded features and mapping the corresponding image pixel values directly to complex-valued channel input symbols. The communication channel is designed as a non-trainable NN layer architecture, and two CNNs parameterize the encoding and decoding functions in their E2E communication system design. Two-channel models are considered in their work, the slow Rayleigh fading channel and the AWGN channel, due to their widespread usage in representing realistic channel conditions. However, if expressed as a differentiable transfer function-based non-trainable neural network layer, various models may be incorporated. Also, by designing an E2E communication framework, Kurka et al. [87] expand on the work of Bourtsoulatze et al. [89] by introducing a JSCC scheme based on autoencoders and employing the channel’s feedback output, named DeepJSCC-f. It significantly enhances the average latency for variable-length transmission or the E2E image quality recovery for fixed-block-length transmission. Utilizing the feedback channel enhances image transmission quality by sending images in multiple steps across multiple layers. Also, adaptable techniques like variable rate transmission significantly optimize resources during communication with desired quality objectives. The ToSC for image transmission is summarized in Figure 6. To accomplish compelling image semantic reconstructions, a deep learning-based image reconstruction technique using a Generative Adversarial Network (GAN) layer is used to recover SI and synthesize a visually pleasing base layer, along with Better Portable Graphics (BPG) residual Coding to improve perceptual and distortion performance. One way to conceptualize the proposed framework [125] is as a refinement module comprising residual Coding, a conditional generator with spatial normalization, and an encoder. Due to the enormous number of networks needed to cover the accurate transmission across a broad spectrum of SNRs, significant storage is required, and computational inefficiency occurs during the training phase. Thus, Xu et al. [60] propose an attention-DL-based JSCC that functions and performs well at various SNR levels during image transmission. The attention strategy optimizes and allocates resources during source coding based on the channel condition. Using E2E training and DL techniques, [126] proposes an extractor of multidimensional semantic features from an image for transmission named MLSC-image. However, Li et al. [127] exploit medical domain knowledge to extract and reconstruct SI from medical images. In contrast, Huang et al. [59] leverage the power of reinforced learning (RL) by proposing an E2E adaptive semantic coding named RL-ASC to encode images at a higher level than pixels.

Audio/Speech reconstruction:Speech transmission has also been the focus of ToSC systems [108]. In [108], an extension for DL-SC speech transmission systems named DeepSC-S was proposed. Specifically, distortion of source and channel effects could be addressed by joint semantic-channel coding. The minimum difference between the input and recovered speech signals is achieved using a loss function, Mean Squared Error (MSE). Furthermore, the quality of speech signals after recovery is measured using two performance metrics: Perceptual Evaluation of Speech Quality (PESQ) and signal-to-distortion ratio (SDR). To cooperatively train the CNN-based encoder and decoder across numerous edge nodes and servers, DeepSC-S is adapted to a multi-user scenario. A loss function, MSE, which barely represents the quantity of SI at the receiver, is employed.

Again, to adapt to a natural practical dynamic environment of the non-ergodic channel while skipping retraining of the model-enhanced DeepSC-S using squeeze-and-excitation (SE) networks as an attention mechanism, the transceiver was designed as an E2E system that learns and extracts the speech SI [108]. To extract and encode SI from the input speech spectrum at the transmitter and recover accurate transcriptions at the receiver, a DL-based transceiver was proposed in [105]. In addition, a redundancy removal module, a soft alignment module, and a CTC alignment module were incorporated to reduce semantic redundancy and extract more speech-related information. WER and Bidirectional Encoder Representations from Transformers (BERT) were adopted for evaluation.

Table 8. Summary of ToSC approaches.

Category	Core Modeling Principle	Optimization Objective	Representative Techniques	Adaptivity Mechanism	Typical Applications
Probabilistic / Information-Theoretic Methods	Information Bottleneck (IB), Rate-Distortion Theory, Mutual Information Minimization	Minimize $I(X;Z)$ while maximizing $I(Z;Y)$; task-aware rate allocation	IB, Variational IB (VIB), Task-oriented rate-distortion optimization	Trade-off control via $\beta$; channel-aware rate adjustment	Task-aware compression, edge inference, bandwidth-constrained learning
Graph-Based Coordination Frameworks	Structured dependency modeling across agents or modalities	Joint semantic consistency and coordinated feature exchange	Graph Neural Networks, GCIB, multi-agent feature fusion	Topology-aware adaptation; dynamic edge weighting	Multi-UAV systems, cooperative sensing, distributed perception
Learning-Based End-to-End Methods	Deep joint source-channel learning; representation learning	Direct task loss minimization under channel constraints	Deep JSCC, CNN/Transformer encoder-decoder, RL-based adaptation	Channel state-aware training; feedback-driven encoding	Wireless image/speech transmission, edge AI systems
Multimodal Foundation-Based Approaches	Cross-modal semantic alignment in shared latent space	Contrastive/generative pretraining with task alignment	Vision-language models, multimodal Transformers, attention-based fusion	Cross-modal attention; semantic prioritization	Vision-language ToSC, embodied AI, multimodal edge intelligence

Table 8. Summary of ToSC approaches.

Category	Core Modeling Principle	Optimization Objective	Representative Techniques	Adaptivity Mechanism	Typical Applications
Probabilistic / Information-Theoretic Methods	Information Bottleneck (IB), Rate-Distortion Theory, Mutual Information Minimization	Minimize $I(X;Z)$ while maximizing $I(Z;Y)$; task-aware rate allocation	IB, Variational IB (VIB), Task-oriented rate-distortion optimization	Trade-off control via $\beta$; channel-aware rate adjustment	Task-aware compression, edge inference, bandwidth-constrained learning
Graph-Based Coordination Frameworks	Structured dependency modeling across agents or modalities	Joint semantic consistency and coordinated feature exchange	Graph Neural Networks, GCIB, multi-agent feature fusion	Topology-aware adaptation; dynamic edge weighting	Multi-UAV systems, cooperative sensing, distributed perception
Learning-Based End-to-End Methods	Deep joint source-channel learning; representation learning	Direct task loss minimization under channel constraints	Deep JSCC, CNN/Transformer encoder-decoder, RL-based adaptation	Channel state-aware training; feedback-driven encoding	Wireless image/speech transmission, edge AI systems
Multimodal Foundation-Based Approaches	Cross-modal semantic alignment in shared latent space	Contrastive/generative pretraining with task alignment	Vision-language models, multimodal Transformers, attention-based fusion	Cross-modal attention; semantic prioritization	Vision-language ToSC, embodied AI, multimodal edge intelligence

2.8.2. Inference Tasks

Intelligent tasks have been made feasible with appropriate interactions with source data and destination decisions, thanks to the development of NLP, CV, and deep learning (DL) techniques. ToSC systems carry out intelligent communication tasks based on the transmission goal. The global semantic features are extracted for transmission to complete the task inference at the destination. The task-oriented semantic transmission system only extracts semantic features meant for the particular task at the moment of transmission, in contrast to conventional communication, which compresses and transmits the entire raw data to the destination for inference, as depicted in Figure 5.

Significant contributions to the advancement of different intelligent tasks have been present, including Machine Translation(MT)[51,52], question-answering (QA)[128], Image classification[129,130], Visual Question Answering (VQA)[129,131], Speech Recognition and Synthesis[48,68], and Time series classification [132]. Therefore, this subsection discusses ToSC in detail for several source data modalities, including textual, visual, audio/speech, and multimodal data.

Machine translation:A ToSC system for machine translation (MT) called DeepSC-MT was introduced in [51] that recovers the accurate semantic meaning of transmitted texts. The model consists of a text semantic encoder for text extraction and a Joint Source Channel (JSC) encoder for compression. The text JSC encoder in Figure 7 is built with multiple dense layers, and the text semantic encoder uses numerous Transformer encoder layers.

Visual question answering: In a VQA task, the model processes the text questions and visuals to determine the correct response. Consequently, most VAQ models use cross-entropy as the loss function and answer accuracy as the performance parameter. Some users send texts to inquire about the details of the photographs, while others send similar images. Using the visual question answering (VQA) task as an illustration, a task-oriented, multi-user ToSC system was proposed in [131] for multimodal data transfer. In particular, Xie et al. [131] present a DNN-enabled ToSC system, called MU-DeepSC, to take advantage of the correlation among the multimodal inputs from different users. The MU-DeepSC transmitter uses a memory, attention, and composition (MAC) neural network to handle the associated data. The MU-DeepSC receiver, per SI, directly generates the replies received after extracting the images and text from various transmitters.

Speech recognition and synthesis: As shown in Figure 8, speech recognition (SR) tasks require the transmission of input speech from the source to the server for the inference task. Depending on the speech content, these tasks are further classified as conversation-type tasks (such as human inquiry) or command-type tasks (such as smart home control). Conversation-type tasks aim to facilitate natural conversations between humans and devices by comprehending language, intent, and sentiment, unlike command-type tasks, which parse specific commands from transmitted speech and control the target device.

We can think of the transceiver as an E2E system and assume a ToSC system for the SR task. Specifically, when a ToSC system is fused with DL, text-related semantic information is learned and extracted at the transmitter, allowing it to transmit far less speech than the source data while maintaining the intended performance [68]. Moreover, it is feasible to design a robust model to handle different channel settings without retraining to facilitate speech recognition for channels with dynamic characteristics [68]. The source data is compressed using a classical CNN, and features are learned and extracted from the input speech sounds using a gated Recurrent Unit-Bidirectional RNN (GRU-BRNN). In this way, the model accurately reconstructs the text transcription by sending only text-related information to the receiver. Furthermore, DeepSC-SR was trained with a fixed channel condition to provide a trained model with solid robustness to channel fluctuations, and was effectively used for various testing channel settings. Later, Weng et al. [48] extended the DeepSC-SR [68] by proposing a speech recognition and synthesis DL-based ToSC system called a DeepSC-ST. A joint semantic-channel encoder first extracts the SR-related semantic features for transmission to significantly lower the necessary volume of utilization without sacrificing performance. The text is then reconstructed at the receiver following the detected semantic features adopted in [68]. Subsequently, speech synthesis is performed at the receiver to generate voice signals by feeding a recognized text to an NN module and speaker information. Furthermore, a reliable ToSC system, called Ross-S2T, is proposed by Weng et al. [133] to accomplish the speech-to-text translation task by providing the necessary semantic information. Specifically, a DL-based semantic encoding module is created to directly condense and translate speech in the original language into semantic features of the text in the target language. This promotes the creation of a DL-enabled ToSC system for E2E training in speech-to-text translation.

ResNet-101 is used as the semantic encoder, and CNNs with various unit configurations are used as the channel encoder to produce the transmitted signals. Next, the first 30 blocks of the ResNet-101 network pre-trained on ImageNet are employed to extract SI in the image semantic encoder. To generate transmitted symbols, the text transmitter uses dense layers with distinct units for the channel encoder and bi-directional long short-term memory (Bi-LSTM) to extract the input sentence’s semantic representations in the semantic encoder. Nevertheless, Wei et al. [134] proposed a federated semantic learning framework useful for task-oriented communications than executing vanilla distributed learning only at the user’s side, allows multiple devices to cooperatively train their semantic-channel encoders under the supervision of a base station-based semantic-channel decoder. The framework was motivated by the IB concept, which aims to extract the associated semantic representation by eliminating unnecessary and unimportant aspects while maintaining task-related data. As a result, while maintaining communication accuracy, the codified rate-distortion tradeoff lowers communication overhead.

Table 9. Challenges and Lessons Learned in ToSC Systems.

Challenge	Observed Limitation	Lesson Learned
Redundant Semantic Representation	Raw signals contain high redundancy irrelevant to downstream tasks	Effective semantic compression must prioritize task-relevant features using IB-driven or task-aware objectives
Compression-Accuracy Trade-off	Excessive compression degrades task performance	Adaptive bottleneck control (e.g., dynamic $\beta$, feature pruning) enables balanced rate-accuracy optimization
Channel Variability and Noise	Performance degrades under dynamic SNR and fading	Channel-aware training and joint source-channel adaptation provide graceful degradation and robustness
Multi-Agent Coordination	Independent transmission leads to redundant or conflicting information	Graph-based coordination and structured feature sharing improve collective efficiency
Cross-Modal Fusion Complexity	Heterogeneous modalities introduce alignment challenges	Attention-based alignment and shared latent spaces enable consistent multimodal semantic representation
Scalability to Foundation Models	Large latent dimensions increase communication cost	Task-driven feature selection and adaptive semantic filtering are essential for scalable deployment
Generalization and Deployment Robustness	Distribution shifts reduce real-world reliability	Continual learning and online adaptation strengthen long-term robustness in dynamic wireless environments

Table 9. Challenges and Lessons Learned in ToSC Systems.

Challenge	Observed Limitation	Lesson Learned
Redundant Semantic Representation	Raw signals contain high redundancy irrelevant to downstream tasks	Effective semantic compression must prioritize task-relevant features using IB-driven or task-aware objectives
Compression-Accuracy Trade-off	Excessive compression degrades task performance	Adaptive bottleneck control (e.g., dynamic $\beta$, feature pruning) enables balanced rate-accuracy optimization
Channel Variability and Noise	Performance degrades under dynamic SNR and fading	Channel-aware training and joint source-channel adaptation provide graceful degradation and robustness
Multi-Agent Coordination	Independent transmission leads to redundant or conflicting information	Graph-based coordination and structured feature sharing improve collective efficiency
Cross-Modal Fusion Complexity	Heterogeneous modalities introduce alignment challenges	Attention-based alignment and shared latent spaces enable consistent multimodal semantic representation
Scalability to Foundation Models	Large latent dimensions increase communication cost	Task-driven feature selection and adaptive semantic filtering are essential for scalable deployment
Generalization and Deployment Robustness	Distribution shifts reduce real-world reliability	Continual learning and online adaptation strengthen long-term robustness in dynamic wireless environments

2.9. Lessons Learned Summary

2.9.1. Lesson Learned for Representation and Extraction

As one of the important insights, semantic representations must capture task-relevant features while filtering out task-irrelevant information to reduce transmission cost. CNNs and Transformers have been highly effective in learning such representations by leveraging their ability to model complex relationships within data. Also, universal representations are limited to maximizing task-specific performance; therefore, modifying representations to meet the demand is important. During the extraction process, approaches such as attention have been very helpful in focusing on the task relevant information of the input. In addition, robust semantic extraction is essential for ensuring system performance under noisy and resource-limited environments. Noise-aware training and adversarial learning are frequently employed to improve the resilience of representations to distortions caused by channel impairments. Too much compression might result in the loss of important semantic information, and too little reduction results in high communication cost; therefore, balancing the trade-off between compression and accuracy remains a challenge. The significance of designing robust representation and extraction methods for improving task-oriented communication systems is also highlighted.

2.9.2. Lesson Learned for Encoding and Decoding

The design of Joint encoding mechanisms is fundamental to the success of ToSC systems. One significant insight is the need for joint semantic coders to focus on extracting minimal and sufficient task-relevant features, effectively reducing redundancy and communication overhead while maintaining task accuracy at the receiver. This is achieved by leveraging recent DNN models, including Transformers and their derivatives, to prioritize meaningful features. Decoding is optimized to reconstruct transmitted features and directly support the downstream task, such as classification or object recognition. Robustness is essential to encoding and decoding, especially in noisy channels. Methods like denoising autoencoders and noise-aware training have significantly mitigated channel impairments. Also, adaptive encoding schemes adjust to channel conditions and task requirements, improving system efficiency. Also, the incorporation of joint training strategies that maximize task communication performance by optimizing encoders and decoders E2E. Achieving scalability across a variety of tasks and guaranteeing the interpretability of the acquired semantic representations remain challenges. This demonstrates the necessity of efficient encoding and decoding techniques to guarantee task accuracy and communication effectiveness.

2.9.3. Lesson Learned for End-to-End Optimization

The ability to jointly optimize every component from semantic encoding to task execution has made E2E optimization an effective framework for developing ToSC systems. One important lesson is that task-specific objectives are guaranteed to be aligned with integrated design, treating the communication process as a single distinct pipeline. To balance resource efficiency and robustness to channel impairments, task-oriented loss functions must be developed. With systems having to constantly adapt to changing channel circumstances and task demands, adaptability has to be considered during design. Robustness to noise and uncertainty can be significantly increased by employing RL, attention mechanisms, and adversarial training. Prioritizing task-relevant information, semantic compression efficiently lowers communication cost without sacrificing system efficiency. Scaling to multitask and multimodal applications and enhancing interpretability and standardization remain challenges. To realize the potential of task communication systems, it emphasizes the significance of integrating ML, communication theory, and domain-specific applications.

2.9.4. Lessons Learned for Data Reconstruction and Inference Tasks

The reconstruction task does not aim for the perfect recovery of the transmitted data; instead, it should prioritize reconstructing semantic content essential for the specific inference task. This allows systems to achieve significant efficiency by focusing resources on task-critical information. Architectures such as VAEs and generative models have enabled efficient and robust data reconstruction, even under severe channel impairments. For inference tasks, a major takeaway is the importance of aligning the reconstruction process with downstream task requirements through E2E optimization. For example, the system bypasses exact data reconstruction in classification tasks and directly optimizes task performance metrics. Reliability in practical scenarios has been enhanced by methods like adversarial training for inference modules and denoising during reconstruction, making them significant for robustness to noise and errors. Applications with limited resources, like those with restricted bandwidth, must balance the trade-off between the computational complexity and feature reconstruction quality.

3. System Design for Task Oriented Semantic Communication

Conventional communication often focuses on high-fidelity data transmission and recovery, ensuring that the data received is as close to the original as possible. The need for more efficient communication methods is growing as resource-constrained communication environments, digital twins, and intelligent applications proliferate [1]. As a result, ToSC systems have been developed, prioritizing data transmission that is specifically relevant to a specific task, which works particularly well when limited bandwidth is available. The information being communicated is required to be compelling and pertinent to the task at hand [135]. A ToSC system’s primary design objective is to convey informative features that are concise yet pertinent to the particular task, using low-complexity encoders to achieve high accuracy and low-latency inference. While the ToSC systems have obvious design objectives, achieving these objectives will present several issues. The first is the high-dimensional input data distribution. Many tasks involve input data stored in high-dimensional spaces, such as audio, pictures, and sensor signals. It is challenging to design an effective encoding scheme that captures the critical characteristics without first knowing the distribution of the data because this data is often complex and uncertain. The second challenge is the uncontrolled, task-specific distortion measure. In contrast to traditional communication methods, which quantify distortion using well-defined metrics like BER or SNR, task-oriented systems need metrics particular to each activity. Due to their reliance on task-specific details like detection precision or classification accuracy, these measures could be more manageable. Creating distortion measurements that are both suitable and computationally viable is a significant task. The third and last challenge is high computational complexity, characterized by the need for complex methods like DNNs to extract and compress pertinent information from high-dimensional data. Reaching the intended low-latency performance might be challenging due to the processing demands, which pose a considerable design challenge in achieving an equilibrium between transmission efficiency and computing complexity.

Several cutting-edge approaches have been applied to overcome these challenges. First, E2E deep learning makes it possible to optimize the communication system as a whole by concurrently learning the optimal feature representations and transmission algorithms. Because of its all-encompassing strategy, the system adjusts to the intricate demands of task-specific requirements and high-dimensional data distributions. Secondly, the optimization targets are rendered tractable by applying variational approximation techniques such as variational feature encoding (VFE). These techniques support effective and efficient learning by approximating complex distributions and intractable measures. Lastly, achieving equilibrium between computational complexity and speed requires careful consideration of neural network architectural optimization. Neural architecture search and model compression are two techniques to construct effective models that provide high accuracy and low-latency inference, making them appropriate for real-time applications. Combined with these approaches, a framework for creating ToSC systems that satisfy the demanding requirements of contemporary intelligent applications is achieved. Therefore, Sections A and B discuss E2E communication design and resource optimization, respectively.

3.1. End-to-End ToSC

To achieve a downstream task, an End-to-end Sc system maximizes the transmission process by focusing on extracting, compressing, and delivering task-relevant information as summarized in Figure 9. Feature extraction uses DL algorithms to extract the most important SI from the input data to reduce dimensionality while keeping important information about the task. Then, to reduce information redundancy and improve transmission efficiency, these features are encoded into a more compact form using a DNN, such as a variational autoencoder (VAE) [136]. Last, the compressed features are transmitted over the noisy communication channel. The transmitted features are decoded at the receiver with the aid of background knowledge (BK) to reconstruct the SI required for task inference.

3.1.1. Feature Extraction

Conventional coding techniques often rely on manually created features, which might be inefficient and require specialized knowledge. However, as ML and DL have emerged, automatic and efficient feature extraction has become more popular. Ensuring the reliability and efficiency of E2E communication depends on feature extraction, which also helps reduce dimensionality, improve the accuracy of data interpretation, and mitigate noise. In the time series classification task, Xiao et al. [137] present a robust temporal feature network (RTFN) that combines an LSTM-based attention network (LSTMaN) with a residual structure comprising several convolutional layers, forming a temporal feature network (TFN). It works as a network for extracting local features from data. The LSTM and attention networks are hybridized in LSTMaN, which comprises two identical layers. Using features collected from several data positions, the LSTMaN network functions as a relation extraction network to find the inherent links between the features.

Yafei et al. [138] present an effective E2E local-global-fusion feature-extraction network called LGFFE for enhanced discriminative feature representation in the classification of remote sensing image scenes. In particular, using the channel and spatial dimensions, global and local features are extracted from a high-level feature map via deep CNNs, along with an RNN attention module that gathers contextual data and spatial layout information for local elements at multiple locations. Each region’s crucial weight is subsequently produced using GRUs, which accept an input sequence of information from picture patches.

3.1.2. Feature Compression

Feature compression is employed in E2E communication to lower the volume of data sent across a wireless network without degrading data quality. To maximize bandwidth throughput, co-optimizing the compression and downstream tasks is essential. Specifically, the IB is a well-known method for characterizing the rate-distortion tradeoff that aims to extract and transmit compact yet informative features with enough information to achieve high-accuracy inference [139]. This is accomplished by minimizing the MI between the input data and the features and optimizing the MI between the inference result and the extracted features. Nevertheless, creating the ideal mapping functions, from source raw input data to feature extraction to the feature compression, transmission, and later inference result at the receiver, represents an unsolvable variational optimization issue involving functional variables. To overcome this problem, the mapping function is parameterized using a DNN, and the parameters are directly learned from the data. In addition, achieving high transmission rates over noisy channels is a fundamental objective of communication system design. Shi et al. [1] introduce a new principle called robust IB (RIB) that combines the goals of relevant information extraction and information transmission rate maximization. This principle enhances the IB rule by maximizing the mutual information between the transmitted and received representations. The tradeoff between the encoded representations’ informativeness and the received representations’ resilience to information distortion is characterized by RIB. It handles the trade-off between reducing communication overhead and enhancing robustness while preserving enough informative data for accurate task inference. Through training end-to-end, the learned variational distribution inference model at the receiver was optimized [1].

3.1.3. Feature Transmission

The number of feature elements received after transmission and the distortion resulting from quantization errors and sensing noise determine the accuracy of the achievable inference. It is preferable to design a task-oriented scheme based on integrated sensing, communication, and computation (ISCC) to fully utilize the limited network resources. To reduce sensing noise and increase communication rate, the sensing and communication stages compete for radio resources. Meanwhile, the communication load is determined by the on-device computing result, which includes the quantity of extracted feature elements and the quantization bits of every component. The downstream inference task in [140] is accomplished by transmitting quantized feature subsets extracted from the real-time sensory data to the edge server. The entire feature vector is formed by stacking all feature subsets. Each device senses the source in a narrow view by focusing on a single angle to obtain high-quality sensory data. An ISCC resource management plan is then developed to optimize inference accuracy after the impacts of sensing noise, communication capacity, and the quality of computing results on inference accuracy are characterized. Due to noise in the channel, multiple wide-view sensory data can be collected to improve inference accuracy. That means every sensor recognizes its source in a broad direction for broad-view object detection.

3.2. Multi-User and Distributed ToSC

As intelligent and distributed applications become more ubiquitous, ranging from collaborative robotics to connected vehicles, the design of ToSC systems must evolve toward multi-user and distributed architectures. Several intelligent agents collaborate and communicate semantic information, demanding flexible, efficient protocols that balance performance, privacy, and resource constraints. Early efforts by Xie et al.   cite 9830752, 9653664 proposed Transformer-based semantic encoders for multimodal, multiuser tasks including image retrieval, machine translation, and visual question answering (VQA). Their DeepSC-IR, DeepSC-MT, and DeepSC-VQA architectures demonstrated robustness to channel impairments, reduced complexity, and improved inference performance by aligning semantic embeddings for collaborative tasks. Similarly, in V2X networks, cooperative multiuser semantic communication was explored, showing how collective transmission can enhance task reliability in connected vehicle contexts [141]. Also, the recent M4SC [142] framework employs a multimodal LLM with Kolmogorov-Arnold Network alignment to support multitask and multiuser semantic communications. It introduces a semantic sharing mechanism that separates public and private semantics across users and improves efficiency and robustness in complex channels. The Lightweight vision model for Multiuser SC System (LVM-MSC) further refines this approach by constructing a lightweight semantic codec and enabling multiuser semantic sharing via broadcast when users share similar semantics, reducing redundant transmissions and improving efficiency [143]. Beyond coding, distributed MARL has been leveraged for covert SC. Here, devices and jammers cooperatively optimize semantic transmission policies (e.g., power control, selective semantic forwarding) in a multi-agent framework to balance performance and privacy under adversarial conditions [144]. Also, the most recent concept of importance-aware source channel coding in multimodal environments has been proposed. Techniques like segment-based semantic partitioning and rate-splitting coded progressive transmission allocate protection based on the significance of semantic segments across multiple users, enabling robust and flexible multicast ToSC systems [145]. Figure 10 summarizes how the fusion of semantic decoder and agents for distributed inference can be integrated into the ToSc communication pipeline along with secured semantic transmission policies.

Figure 11. Federated edge learning for ToSC.

Figure 11. Federated edge learning for ToSC.

3.3. Resource Optimization

3.3.1. Edge Inference Enabled ToSC

Shifting DL inference and training from cloud servers to edge nodes has made edge inference a competitive alternative. However, deploying large DL models at the edge incurs significant communication overhead due to limited interaction among edge servers at wireless network access points, base transceiver stations, and edge devices. Edge AI integrates pre-trained DL models with edge computing and makes real-time decision-making and low-latency responses feasible. This reduces bandwidth use during transmission, lessens computational load on cloud infrastructure, and improves privacy on the source device by local inference. New applications deploying AI also pose new communication challenges in the interim [22,23]. However, there are three common ways to minimize challenges in edge inference. The first one is offloading the task to the nearest server for computation and inference (referred to as edge server inference), the second one is computing on specific edge devices (known as on-device inference), and the last one is breaking an extensive DNN model into two components. An edge device hosts computationally efficient, earlier layers of the model, while the edge server hosts the remaining, larger, and more computationally demanding layers; this approach is referred to as edge-split inference or device-edge co-inference. Due to resource constraints, on-device inference offers only prohibitive accuracy [146]. Besides, DNN-based mobile applications exploit the nearest and more powerful edge servers for inference and operate at the edge of wireless network communication to offer a rich experience to users [23,147]. However, enabling latency-sensitive applications, such as self-driving vehicles, for edge server inference imposes a high communication cost and data privacy concern. Thus, limited processing resources and induced communication costs hinder server-based and on-device inference. As a result, the device-edge co-inference approach successfully eliminates the previously identified computational and communication barriers [139,148]. AI-enabled edge devices use local inference to extract relevant features immediately appropriate for the task, including object detection, recognition, or classification. Although it comes at a substantial computational load, processing data at the edge device reduces latency and the need for high-bandwidth transmission. Although it is important when real-time decision-making in applications like intelligent surveillance, and autonomous driving is of high demand. Deploying DNN-based apps on mobile/edge devices is challenging since they usually involve a lot of computation. To improve on-device inference, Yi et al. [149] proposed an on-device inference designed for the mixture-of-experts (MoE) named EdgeMoE. A prominent variation of sparse LLMs that exhibits almost constant computational cost, despite scaling in the number of parameters, has been the first on-device inference engine designed for MoE LLMs. Strategically dividing the model throughout the storage hierarchy allows EdgeMoE to achieve computational and memory efficiency [149]. However, deploying traditional DNNs on resource-constrained edge devices is challenging since they involve substantial computation in both performance and energy. Owing to the intractable long-distance network delay, offloading DNNs to the cloud for execution experiences inconsistent performance due to bandwidth constraints. To solve this, Li et al. [148] introduced hybrid compute resources for a real-time synergistic cooperative on-demand DNN co-inference framework named Edgent. Specifically, Edgent divides DNN computation between the edge device and the edge server and minimizes computation latency by accurately sizing the DNN layers to expedite inferences via early exit at an appropriate intermediate DNN layer. To improve high accuracy given sufficient information, edge split inference aims to extract and transmit compact yet informative features. In particular, the IB has been a well-known method for characterizing the rate-distortion tradeoff between inference and the informational accuracy performances of the encoded feature [139]. To facilitate low-latency cooperative inference, Tarimo et al. [150] and Shao et al. [61] minimize redundant data and send only the information necessary for downstream inference tasks, rather than restoring data samples at the edge server. A learning-oriented communication scheme offers task-oriented optimization for distributed feature encoding and local feature extraction. Besides, [61] adopts IB to extract the features pertinent to the task from every edge device and utilizes the distributed IB framework to formalize a one-by-one letter description of the ideal tradeoff between rate and relevancy for the encoding of distributed features. DNN-based mobile applications exploit edge inference capability and operate at the edge of wireless network communication to offer a rich experience to users. This minimizes the extra delay caused by forwarding raw encoded data to the Cloud [23]. When uploading large amounts of data, like HD videos and 3D graphics, transmission delay still presents a challenge for applications with stringent latency requirements. Since the IB optimization is computationally bound for the data with high dimensions, the variational IB (VIB) [23] has shown a tractable higher limit. To lower the communication cost, Authors in [23] adopt a distribution that induces sparsity as the variational baseline for the VIB framework, enabling it to break the encoded feature vectors into smaller segments. In addition, a dynamic NNs-based variable-length feature encoding approach [23] has adapted the activated encoded feature vector dimension to dynamic channel conditions analogous to real-world communication systems.

Algorithm 1 Hierarchical SCC-RA resource allocation

Input:N, M, $\tau$, R, $\alpha$, $\gamma$, $\mathcal{D}$, B, $C_{\max }$, $W_{\max }$ Initialize: Q-table $Q(s_t^i,a_t^i)$ for each agent with random weights and replay buffer $\mathcal{D}$. Obtain initial global model $w^{\left(0\right)}$. for each global communication round $g=1:R$ do     Cloud Server: Broadcasts initial model $w^{\left(g\right)}$ to all edge servers.     for each edge server $m=1:M$ (in parallel) do         Edge Server: Disseminates model $w_m^{\left(r\right)}$ to its cluster.         for each local training round $t=1:\tau$ do            for each device $i\in {\mathcal{N}}_m$ (in parallel) do                Sensing: Perform sensing task and collect data $d_i\left(t\right)$.                Computation: Allocate computational resources $c_i\left(t\right)\le C_{\max }$ for local processing.                Communication: Allocate bandwidth $b_i\left(t\right)\le W_{\max }$ for data transmission.                Train local model $w_m^{\left(r\right)}$ for $\tau$ steps using $d_i\left(t\right),c_i\left(t\right)$, and $b_i\left(t\right)$.                Select action $a_i\left(t\right)=(c_i\left(t\right),b_i\left(t\right))$ using $ϵ$-greedy policy.                Observe new state $s_{t+1}^i$, store transition $(s_t^i,a_t^i,R_t^i,s_{t+1}^i)$ in $\mathcal{D}$.                Sample minibatch from $\mathcal{D}$ and update $Q(s_t^i,a_t^i)$ $$Q(s_t^i,a_t^i)\leftarrow R_t^i+\gamma \underset{a^{\prime }}{\max }Q(s_{t+1}^i,a^{\prime })$$            end for            End Devices: Send local models and resource usage reports to edge server m.            Edge Server: Aggregates local models. $$w_m^{\left(t\right)}=\sum _{i\in {\mathcal{N}}_m}{\displaystyle \frac{n_i}{N_m}}{w}_i^{\left(t\right)}$$            Edge Server: Disseminates aggregated model to its cluster.         end for     end for     Edge Servers: Send aggregated models and resource usage summaries to the cloud server.     Cloud Server: Aggregates models from all edge servers: $$w^{(r+1)}={\displaystyle \sum _{m=1}^M}{\displaystyle \frac{N_m}{N}}{w}_m^{\left(T\right)}$$     Decay exploration $ϵ\leftarrow \alpha ϵ$. end for Output: Resource allocation policies ${\pi }_c^{*}$, ${\pi }_b^{*}$, and final global model $w^{(R+1)}$.

3.3.2. Federated Edge Learning for ToSC

Recently, the DNN model has shown remarkable results when trained on edge devices via federated learning (FL). Wireless FL incorporates the entire model training at the network edge nodes while maintaining data privacy and minimizing communication overhead [151]. This enhances the exchange of model parameters while retaining confidential data in its original form on the local devices [152]. To enhance privacy protection, Li et al.[153] design a complex feature augmentation strategy that moves actual features closer to the decision border to allow synthetic data to enhance model generalization and remove private feature information.

With DL methodologies and wireless sensing technologies, recent years have seen a surge in intelligence applications, ranging from intelligent healthcare provisioning to autonomous home environments. Federated Edge Learning (FEL) emphasizes privacy-preserving collaborative intelligence at the network periphery [1]. This is poised to serve as the foundational mechanism driving communication intelligence. To maximize performance under the stringent, limited resources of edge devices, a thorough task-oriented co-design approach is required. This is because the three components of FEL—sensing, compute, and communication form a complex interdependent framework. A fundamental challenge still exists because there is no reliable theoretical framework that defines the complex interactions between sensing, computation, and communication resources and how these interactions affect FEL efficiency.

In this discussion, we investigate the problem of optimizing the joint allocation of sensing, computation, and communication resources for FEL using wireless sensing in ambient intelligence environments. Building on this insight, we formulate an optimization problem to accelerate FEL convergence while satisfying constraints on training duration, energy consumption, and sensing accuracy at edge devices. The solution involves a two-step approach: first, optimize the allocation of sensing and communication resources to maximize the number of usable sensed data samples over the entire training process; second, distribute these samples across communication rounds to determine batch sizes that enhance convergence speed. To simplify the first step, we reformulate the joint sensing and communication resource allocation problem into a single-variable optimization problem by leveraging the inherent relationships between control parameters.

Specifically, following data sensing, five steps are proposed to build an FEL in general, including disseminating the global model to edge devices, training the local model at edge devices, aggregating the edge device model at the edge server, updating the edge device model n times, and aggregating the edge model at the cloud server. A critical performance-limiting phase for implementing scaled FEL is the global model aggregation, which combines double compression of local model parameters, transmission, and restoration.

As shown in Figure 12, this framework aims to train DNN models for a communication task using the information collected by edge devices. The edge server coordinates the collaborative training of a shared model, denoted by $w\in {\mathbb{R}}^d$, across all edge devices. If $\xi$ represents the batch sample random seeds, and $l_{(}i,j)(w;\xi )$ local loss function of $i^{th}$ edge device corresponding to $j^{th}$ edge server then our goal is to identify a model w that is capable of minimizing the objective function,

$$\begin{array}{c}\hfill L\left(w\right)={\displaystyle \frac{1}{N}}\sum _{(i,j)\in N}{\mathbb{E}}_{\xi \sim P_{(}i,j)}[l_{(}i,j)(w;\xi )],\end{array}$$

(1)

As shown in Figure 12, the training process is an iterative procedure that spans several communication cycles. Eight primary steps are carried out in arbitrary rounds $(r,R)$, named global model dissemination, local model training, local model updating, local model compression and transmission, edge model aggregation, edge model updating, edge model compression and transmission, and global model aggregation, explained as follows.

Global Model Dissemination: The global model parameters $w_g^R$ are broadcast to all participating edge devices by the cloud server. Local training on each client begins with this global model, either randomly initialized or pre-trained. The server guarantees consistency in the learning process throughout the network by providing a standard model to every device. The edge devices download the global model $w_g^R$ distributed from the cloud server over the wireless broadcast channel at each global round R.

Local Model Training and Updating: At round $(R$, each edge device $i,j$ starts local training using its data of a batch size of $b_{i,j}^R=\left|D_{i,j}^R\right|$ after obtaining the global model $w_g^R$ from the cloud server. As a result, the local model $w_{i,j}$ can adjust to the distinct features of the local dataset, guaranteeing customized enhancements. If $n=0,2,....\tau -1,\forall i$, $\eta$ represents the learning rate, $g_i\left(w^{(R,n)}\right)={\displaystyle \frac{1}{b^r}}{\sum }_{m\in D_i^R}\nabla l\left(w^{(R,n)}\right);x_m,y_m$ Stochastic Gradient then every edge device updates its model by applying SGD in $\tau$ steps starting from $w_g^R$ and sending to the edge server as follows;

$$\begin{array}{c}\hfill w_{i,j}^{(R,0)}=w_g^R,\end{array}$$

(2)

$$\begin{array}{c}\hfill w^{R,n+1}=w^{(R,n)}-\eta g_{i,j}\left(w^{(R,n)}\right)\end{array}$$

(3)

Model Compression: The size of models and frequent communication caused by FEL’s requirement for several edge devices to submit model changes (weights or gradients) to an edge server lead to serious bandwidth and latency problems, especially with limited resources. A large model size during training could require an impractical data volume when transferred between the local devices and the edge servers. Compression is incorporated to address this challenge to create sparse models from large networks with much-reduced bandwidth and storage needs, analogous to distributed lossy source coding problems in conventional compression techniques[154]. However, traditional compression techniques do not take advantage of the unique features of FEL tasks, such as the learning algorithm pattern, model aggregation structure, and local models’ spatiotemporal covariance, because their primary objective is usually to minimize the volume of data to be transmitted while allowing some degree of distortion. In this case, model compression in FEL is carefully balanced to avoid significant loss. The trade-off between improving model compression and maintaining model robustness while ensuring convergence across non-IID data is carefully considered to depict the long-term effects on FEL performance.

Sparsification and quantization have been proven methods for substantial model compression in FEL by leveraging local model information structures to minimize transmission overhead and attain optimal learning performance [154]. However, for uplink transmission, the parameters resulting from sparsified models are further projected into low-dimensional signals, followed by compressed sensing to reconstruct the aggregated edge model. Specifically, combined vector quantization and sparsification are used to lower communication overheads further. An adaptive clustering approach for hierarchical FL [155] is adopted, which divides clients into clusters and finds a core client. Moreover, to improve transmission efficiency, compressed models from other clients in the same cluster are gathered by each core client, which then applies a local aggregation with a compression technique. This technique leverages the clustered FL to optimize the communication resources. With the help of Neural Architecture Search (NAS) algorithms, model compression can be enhanced by automating the design of compact neural networks that preserve excellent performance while using fewer resources, especially for edge devices [156]. Moreover, NAS facilitates multi-objective optimization, balancing accuracy with compression goals, which is a double benefit. Also, the integration of model pruning, quantization, or knowledge distillation further reduces model size without degrading performance. Using an effective architectural search method known as robust neural temporal search (RNTS), it is feasible to extract hidden and informative shapelets in ToSC tasks such as Time series classification [157]. Further integration of Differentiable Neural Architecture Search (DARTS) with RNN cells to stabilize skip connections, particularly for time series data, is a promising approach. In particular, designing a more robust model compression encoder for better temporal information extraction and a noise-adding scheme enhances reliability in harsh and dynamic channel conditions compared to traditional methods [158].

Model Transmission: After $\tau$ local updates, each edge device enters transmission mode and uses the uplink wireless channel to upload its local model $w_{i,j}^{(R,r,\tau )}$ as a baseband signal $x_{i,j}\left(t\right)$ to the edge server. Since every device occupies its separate upload frequency subcarrier in a noisy communication channel, and different uplinks transmit normalized power, $p_{(u,i,j)}$, then at time t, the baseband signal received from the ${(i,j)}^{th}$ device through the communication channel $h_{i,j}\left(t\right)$ with AWGN noise, $n_{i,j}\left(t\right)$, can be expressed as;

$$\begin{array}{c}\hfill y_{i,j}\left(t\right)=\sqrt{p_{(u,i,j)}\left(t\right)}{h}_{i,j}\left(t\right)x_{i,j}\left(t\right)+n_{i,j}\left(t\right).\end{array}$$

(4)

Fast Rayleigh fading channels have a larger channel coherence time. The fast Rayleigh fading channel capacity $C_{i,j}$ between the ${\left(i,j\right)}^{th}$ edge device and the ${\left(j\right)}^{th}$ edge server with bandwidth $W_0$ and noise spectral density $N_0$ is quantified as a function of $p_{(u,i,j)}$ based on the ergodic capacity of particular the ${\left(i,j\right)}^{th}$ edge device as:

$$\begin{array}{c}\hfill C_i\left(p_{(u,i,j)}\right)={\mathbb{E}}_{(h,i,j)}\left[W_{0}lo{g}_2\left(1+{\displaystyle \frac{p_{(u,i,j)}{\left|h_{i,j}\right|}_2}{W_0{N}_0}}\right)\right].\end{array}$$

(5)

State of the art in task-oriented wireless transmission over federated edge learning strategy offers optimal efficiency; however, several significant challenges remain. First, identifying the most relevant local model updates for a particular task is complex, particularly in non-IID data environments, where data distributions across edge devices vary significantly. Second, balancing communication efficiency with model accuracy is critical, as aggressively filtering updates could lead to suboptimal global model performance; lastly, implementing a robust task-oriented federated edge wireless transmission framework across heterogeneous devices with varying computational power and network capabilities needs to be addressed. To address learning-aware resource allocation for scheduling devices, a task-oriented model transmission approach with integrated communication and computation has to be developed to scale up the model transmission. Integrating model compression with forward error correction and re-transmission into a conventional lossy wireless FL network [159] is another method to address this challenge.

Edge model updating: To ensure edge model improvement, the edge server j receives edge updates $w_l^{(R,r)}$ from the local device at round $R,r$, then aggregates and locally disseminate the edge model $w_e^{(R,r)}$ to the local devices with weight state s. The local devices repeat the process (2) for z steps as follows;

$$\begin{array}{c}\hfill w_{i,j}^{(R,r,s)}=w_e^{(R,r)},\end{array}$$

(6)

$$\begin{array}{c}\hfill w^{R,r,m+1}=w^{(R,r,m)}-\eta g_{i,j}\left(w^{(R,r,m)}\right)\end{array}$$

(7)

such that, $m=0,1,....z-1,\forall i,j$

Model Aggregation: The edge model, a particular aggregation function of local models, is updated by the edge server throughout each communication loop in round r. Upon receiving the models from every device, the server compiles them to build an updated edge model $w_g^{r+1}$ such that,

$$\begin{array}{c}\hfill w_g^{(R,r+1)}={\displaystyle \frac{1}{N}}\sum _{i\in N}{w}_{i,j}^{(R,r,\tau )},\end{array}$$

(8)

However, the local models are typically high-dimensional, and an extensive number of edge devices participating in FEL would result in a high communication load and an infeasible training duration. For task-oriented FEL communication learning, a multi-access yet ultra-fast model aggregation strategy is implemented to optimize the global server function, for example, over-the-air computation [160]. To improve training for non-ergodic channels, in every local model training and communication loop, we use the MSE loss as the aggregation indicator to quantify local model performance [161] and assign proportional weights for dynamic global aggregation. Including these scenarios in parallel with heterogeneous data distributions (non-IID data environments) from edge devices in time-varying environments, the model aggregation function is designed appropriately to improve its robustness and accuracy, allowing for enhanced global model performance.

3.3.3. Problem Formulation and Optimization

We consider an FEL framework consisting of multiple clustered edge devices engaged in collaborative model training as shown in Figure 12. In particular, this trains a global model while sharing limited wireless and computational resources. The primary challenge here is to optimize the joint allocation of sensing, computation, and communication resources to accelerate FEL convergence for efficient communication.

We formulate the problem as a Markov decision process, and each edge device i is an agent interacting with the environment to make optimal resource allocation decisions. Wireless channel conditions $h_i$, communication bandwidth $b_i$, sensor accuracy levels at each edge device, and available computational power $p_i^c$ are all included in the state space $S_i$. Three significant resource allocation choices make up the action space A. Adjusting the precision and sensing frequency $f_i$, allocating CPU resources for local training, and optimizing bandwidth allocation for model aggregations. The reward function is designed to maximize FEL convergence while minimizing latency and ensuring adequate sensing accuracy. Specifically, we define the reward as follows:

$$R(s,a)={\lambda }_1{f}_{\mathrm{conv}}-{\lambda }_2{C}_{\mathrm{latency}}-{\lambda }_3{C}_{\mathrm{accuracy}},$$

(9)

, $f_{\mathrm{conv}}$ is FEL convergence rate, $C_{\mathrm{latency}}$ measures computation and communication delays, and $C_{\mathrm{accuracy}}$ quantifies the sensing accuracy deviation from an optimal threshold.

The constraints of the FEL framework include a computational resource limit per device, a total bandwidth availability, and a minimum sensing accuracy requirement. Our objective is to design a multi-agent policy that learns optimal resource allocation decisions and is capable of adapting to variations of wireless channels. As shown in Algorithm 1, inspired by Multi-agent Deep RL [165], we propose a hierarchical sensing, computational, and communications resource allocation (SCC-RA) strategy to optimize resource allocation for FEL, such that each edge device acts as an autonomous agent. Specifically, input settings to the SCC-RA framework include the number of participating end devices N, the number of edge servers M, local training steps $\tau$, edge aggregation rounds r, global rounds R, learning rate $\alpha$, discount factor $\gamma$, replay buffer $\mathcal{D}$, batch size B, Maximum computational resource $C_{\max }$, and maximum bandwidth $W_{\max }$.

3.4. Performance Comparison on Different Tasks

Recent studies have introduced a diverse set of methods tailored to various downstream tasks, aiming to maximize inference accuracy at the receiver and minimize communication channel usage. In this subsection, we review and categorize representative works according to the tasks each supports, including classification, recognition/detection, and multitask learning. Each has been compared in terms of architectural design, semantic encoding strategy, and performance under communication constraints. The brief summary of NN architectures such as ResNet [171], LSTMs [172], and Transformers [173] along with their derivatives designed and trained from diverse datasets to minimize communication cost and latency in downstream tasks is summarized in Table 10, Table 11 and Table 12.

Figure 13. Classification task accuracy gain comparison across multiple benchmark datasets.

Figure 13. Classification task accuracy gain comparison across multiple benchmark datasets.

Figure 14. Accuracy gain comparison for recognition tasks across diverse benchmark datasets.

Figure 14. Accuracy gain comparison for recognition tasks across diverse benchmark datasets.

3.4.1. Classification Task

Using the IB framework, Hu et al. [35] designed a fundamental framework that preserves only the features relevant to the specific task for transmission. The framework works well in controlled settings; however, when tested under varying wireless channel conditions, the static encoding approach is less reliable. To address this, Chen et al. [177] proposed a rate-controllable encoder employing a dual-branch network: one path captures semantic features while the other dynamically adjusts the compression level via a learnable controller informed by instantaneous CSI. This design achieved a 35% reduction in transmission cost with only a 2.1% classification accuracy degradation under fluctuating SNRs (5–25 dB) on the CIFAR-100 dataset. To reduce dependence on large-scale labeled datasets, a common bottleneck in SC training, Gu et al. [178] introduced a task-oriented low-label SC framework leveraging self-supervised learning (SSL). The system incorporates a contrastive pretext task during pre-training to enhance semantic feature extraction without requiring extensive annotation. The method is especially well-suited to limited resources because only a small portion of labeled samples is required for fine-tuning during the downstream classification task phase. Experimental results on CIFAR-10 and Tiny-ImageNet demonstrate that the SSL-based encoder outperforms fully supervised baselines under low-label regimes by up to 7.8% in classification accuracy and maintains comparable robustness across SNR ranges of 0–20 dB. Also, the method achieves a notable reduction in transmission cost (up to 28% fewer bits) through selective semantic representation, without sacrificing task performance. Aforementioned works mark a transition from channel-agnostic compression to channel-aware and task-adaptive learning, enabling reliable classification under diverse communication environment conditions. Common datasets to evaluate the recent progress in classification tasks include MNIST [179], CIFAR10 [180], and TinyImageNet [181] Classification works based on CIFAR10 [182], multiview (MNIST, CIFAR10 and TinyImageNet) [61,183,184,185]

3.4.2. Recognition Task

Recognition problems (object detection, segmentation, and 3D perception) require semantic encodings that preserve spatially structured information and remain robust to channel impairments. Recent studies converge on three practical principles for recognition-oriented ToSC: region-aware encoding, joint source–channel representations, and distributed coordination. To assign bits when task accuracy demand is highest (such as object bodies and bounds), region-aware encoders employ attention methods. Task-oriented video and multi-view approaches use spatiotemporal feature selection to reduce bitrate with minimal loss in detection metrics compared to pixel-centric schemes, demonstrating that selective protection of salient regions significantly increases accuracy per bit [97,186]. Because Deep JSCC and its derivatives learn continuous feature-to-channel mappings optimized with recognition losses, they eliminate the brittle separation between compression and error control. When channel conditions deteriorate, current models outperform classical separated pipelines on recognition benchmarks and exhibit acceptable task-level degradation at low SNRs [86,87,187]. On the other hand, distributed and collaborative multi-sensor systems can maintain collective recognition performance in parallel with lowering aggregated transmission bits through minimizing mutual information among correlated sensor encoding [97,186]. This method is particularly effective in vehicular and UAV perception. Related recognition work includes human motion recognition tasks [188,189,190] using a dataset from PBAH [191].

3.5. Applications and Use Cases

As intelligent technologies continue to be incorporated into existing communication systems, the deployment of ToSC has grown across a spectrum of industries, for example, manufacturing, transportation, healthcare services, UAV communication, remote sensing, etc. The adoption of ToSC has grown across industries such as manufacturing, transportation, healthcare, UAV communication, and remote sensing, fueled by the increasing integration of intelligent technologies into existing communication systems. Transmitting only task-related SI increases the efficiency of information transmission and streamlines machine-to-machine (M2M) and human-to-human (H2H) interactions. To overcome communication issues in these applications, recent research investigates efficient frameworks for task-related SI extraction methods, adaptive joint feature coding, and optimization techniques. The potential contributions of ToSC are highlighted in this section, and it offers a thorough summary of how it is used in five areas.

3.5.1. Intelligent Transportation Systems (ITS)

The ultra-low latency and real-time data transmission between vehicles, roadside infrastructure, and cloud-based control systems in intelligent transportation systems (ITS) have been fueled by 5G wireless communication [192]. The advent of 5G V2X further empowers complex functionalities such as dynamic route optimization, cooperative autonomous navigation, large-scale traffic state estimation, and proactive collision mitigation [193]. High-definition maps, sensor readings, and vehicle status data are only a few of the diverse data generated by connected vehicles and are essential for AI-powered decision-making [194,195]. Under this condition, there is both high communication latency and high bandwidth consumption due to a massive amount of data being generated. Due to dynamic channel in high mobility environemnt, transferring raw vehicle data in real time via dynamic vehicular networks presents considerable chellenges in real-time decision making necessitates extremely low latency. Also, due to limited processing power and energy constraints, executing complex tasks directly on individual vehicles results in computational bottlenecks[196]. Vehicles can only transmit extracted SI relevant to object recognition to reduce transmission costs and increase communication efficiency, hence addressing the issues encountered by autonomous driving systems [197]. With a multi-task-oriented communication architecture, the front-end vehicle models encode and transmit image scenes such as road traffic signs semantically and efficiently [198]. In particular, the framework uses convolutional autoencoders to extract and transmit image features from one connected and autonomous vehicle (CAV) to another for image classification or image reconstruction tasks. The approach saves up to 89% of the data compared to traditional communication JPEG+QAM 16, significantly improving robustness and efficiency in multi-user ITS scenarios. Since reliability is essential in vehicular communication, Zhou et al. [199] design a model that minimizes execution time and estimates the likelihood that an application will complete given its limited computational capabilities and requirements in a cooperative environment. Specifically, constrained optimization models optimize the data partitions among various cooperators to increase computation- and communication-related reliability. When transmitting 5 Mbits of data from the vehicle to the infrastructure, the cooperative computing modes, and the cloud computation, the vehicle-to-infrastructure (V2I) uplink has a nearly 100% non-outage probability with a 95% probability of application completion with greater reliability cite 8674597. This demonstrates the effectiveness of cooperative computation that is optimized for reliability in supporting the execution of remote applications. Delay minimization is essential for V2I communication. Using the Lyapunov optimization method, Ning et al. [200] design an online multi-decision-making strategy to minimize communication delay edge computing in the Internet of vehicles. In particular, edge intelligence-enabled IoV with a hierarchical architecture that takes into account content caching, infrastructure peer offloading, and V2I computes offloading. Next, an optimization problem using Mixed Integer Non-Linear Programming is used to reduce the overall network latency [200].

3.5.2. Intelligent UAV Communication

In 5G and beyond wireless communication, Unmanned Aerial Vehicle (UAV) communication has become an attractive research topic. UAVs are now capable of being used in surveillance, road and building segmentation, search operations, and object tracking, to mention a few. A variety of data, such as position data, sensor outputs, and images, are produced by UAVs and are essential for operations powered by AI. Because of the frequent variation in channel characteristics, especially when the ground user is moving, transferring raw generated data in real-time over a non-terrestrial network presents issues with reliability and latency[201]. In addition, because of compute and power limitations, the processor creates a bottleneck when the complete work is carried out at each UAV node [202]. The ToSC has been envisioned as a promising solution. For example, in surveillance operations, instead of streaming large volumes of high-resolution video or full sensor datasets, UAVs may only need to transmit information about specific object features for object recognition or object classification tasks, drastically reducing the data overhead and improving communication efficiency. To achieve low latency and high reliability in cooperative UAV swarm systems, multiple UAVs work together towards a common objective. Yun et al. [203] propose a graph attention exchange network, a multi-agent deep RL approach to combat the reliability and latency challenges of air-to-ground communication. In particular, each UAV locally builds an attention graph to assess its focus on neighboring UAVs while sharing semantic weights with others to minimize discrepancies in attention levels among them. For centralized training and distributed execution, a neural network is employed to maximize the reward for joint action between UAVs. The joint action obtained guides each UAV to converge to its local optimal function. The less latency requirement of about $28.8\mu s$ for a payload of 576 bits and 20 MHz bandwidth is obtained, which falls within the range of 5G transmission latency requirements [204]. For adaptive communication mechanisms that efficiently utilize bandwidth in UAV segmentation tasks with semantic communication capabilities, UAVs dynamically adjust the data transmitted based on available bandwidth, sensor status, or channel conditions. This dynamic adjustment allows UAVs to prioritize transmitting saliency segmentation features by compressing less relevant information, ensuring more efficient utilization of available communication resources. Specifically in UAV-captured aerial imagery, a ResNet-based framework is utilized to extract essential features [205]. These features, along with the elevation map, are then processed through a spatial dependency network that employs a Convolutional Bi-LSTM. The refined output from this stage is passed into a Semi-Supervised Convolutional Autoencoder. The encoder’s output undergoes convolution and softmax operations to produce a segmentation map, while the decoder reconstructs the input features for further small object segmentation. Despite transmitting scene-extracted features, the framework achieves competitive results of 0.845 and 0.8157 for mean precision and mean recall, respectively [205]. To achieve optimal computation for resource-constrained UAVs, a computationally efficient model is deployed on the UAV’s front end to enable scene understanding and transmission, integrating a semantic extraction module with dynamic channel state awareness [206]. Furthermore, an extension of the deep RL framework is employed to optimize the trade-off between classification precision and transmission latency. Thus, the framework selects the most informative semantic blocks, maximizing their mutual importance to the mobile edge computing classifier under dynamic wireless channel conditions. Besides, for UAVs with limited storage capability, tiny ML enables UAVs to execute sophisticated algorithms on energy-efficient, resource-constrained hardware. In particular, a framework that aligns heterogeneous multimodal data transmission with UAV task execution, exploiting the capabilities of ML and semantic encoding, was proposed to optimize task-oriented UAV operations [207]. Multimodal virtual semantic communication facilitates cross-modal data integration, enhancing task-related inference through complementary modality fusion. The developed model establishes a tightly coupled architecture that unifies communication, perception, and computation, significantly improving the UAV’s operational efficiency in dynamic environments [207].

3.5.3. Smart Manufacturing

In the Fourth Industrial Revolution, communication optimization is essential in smart factories, enhancing real-time monitoring and efficient collaboration among robots and machines. Latency sensitivity in practical processes, data heterogeneity from various sensors and devices, and resource constraints in energy-constrained IIoT systems emerged as key challenges in traditional industrial IoT (IIoT) communication. Communication methods in large-scale settings are limited by constrained bandwidth, increased latency, and computational overhead due to the large amount of sensor data generated. Mostaani et al. [208] propose a scalable task-oriented communication design framework that leverages RL and the VoI to optimize data transmission in cooperative multi-agent systems (MAS). The system models agent interactions using Markov Decision Processes (MDPs), and agents receive observations, select actions, and aim to maximize their long-term task rewards. The optimal communication strategy is identified based on how data contribute to achieving the task objective. The VoI is computed using Q-learning, which updates a policy by maximizing expected rewards based on past interactions. The system applies the learned VoI to determine optimal quantization policies, ensuring only high-value data is transmitted. After that, task-relevant features are retained when high-dimensional sensor readings are compressed into a compact representation through the quantization process. To reduce transmission cost without compromising accuracy, a clustering-based approach divides sensor data into discrete communication messages according to saliency. Each agent follows a bit-budget constraint, ensuring that communication remains scalable as the number of agents increases. Learning the VoI and employing intelligent quantization policies significantly reduce the computational complexity associated with multi-agent communication while maintaining high system performance. Since privacy and communication efficiency are essential, Yang et al. [209] design a distributed k-nearest neighbors (kNN) classification algorithm for reliable defect detection and anomaly identification in smart manufacturing. To protect data privacy while maintaining classification accuracy, the method incorporates vector homomorphic encryption (VHE), allowing computations to be performed on encrypted data. Also, the use of the Map/Reduce framework enables efficient processing of large-scale encrypted data across multiple servers, making it suitable for Industrial IoT (IIoT) applications.

3.5.4. Remote Sensing

In remote sensing applications of task-oriented communication such as geographic data collection, urban planning, and road measurement, data sources are often heterogeneous, consisting of diverse modalities such as imagery, environmental measurements, and geospatial data. In land cover classification, probabilistic models help generate semantic features, but they often face challenges with adaptability. To address this, a hierarchical multi-modal probabilistic latent semantic analysis (PLSA) model [210] has been introduced. Specifically, this model utilizes multiple lexical modes and topic levels to enhance semantic representation across different sensors, particularly for integrating synthetic aperture radar and multispectral imaging data. Building on this approach, an unsupervised land cover classification method extracts multi-modal features from complex images with limited available information. By improving image fusion and classification accuracy, this method advances semantic communication in remote sensing, making it more adaptable and effective for diverse environmental and geographic applications. For multi-modal remote sensing data, an adaptive fusion network has been developed to enhance the integration of global and local contextual features, improving the overall observation of remote-sensing images [211]. To optimize land cover identification, a fusion architecture search framework based on multi-modal semantic consistency has been introduced. In particular, the framework facilitates effective feature extraction and fusion by employing gradient optimization to explore the feature space while incorporating semantic consistency constraints to assess feature correlation.

4. Future Research Directions And Open Issues

Integrating large-scale multimodal foundation models into semantic encoders and decoders is one of the potential approaches that allows for zero-shot task adaptation without requiring extensive retraining. If trade-offs between knowledge retention, model compression, and on-device adaptation are appropriately managed, the approach may enable systems to adapt dynamically to new tasks and channel conditions using pretrained knowledge. For ToSC, designing frameworks for continual learning presents another substantial challenge. It should be possible for future systems to accumulate knowledge over time, incorporate new task requirements, and selectively discard outdated information. To ensure low computational and communication costs, such adaptability will necessitate developments in parameter-efficient fine-tuning algorithms, federated continual learning, and semantic knowledge distillation. In multi-agent systems, semantic communication should facilitate cooperative reasoning and decision-making, going beyond a perception-centric task approach. To improve coordination among agents, transmitted information between cooperative distributed agents needs to include contextual and causal explanations in addition to task-relevant observations. This will require more efficient representations that are interpretable, task-oriented, and robust to partial observability and channel noise. Last but not least, the integration of ToSC schemes into programmable and AI native 6G infrastructures presents both opportunities and design challenges. For example, the combination of reconfigurable intelligent surfaces (RIS), digital twin-assisted optimization, and cross-layer co-design may enable semantic encoders to adapt to variations in channel state and network topology. To achieve these improvements and guarantee comparability across various methodologies, standardized benchmarking suites to cover a variety of modalities, SNRs, and inference task types will be required. Similarly, an emerging research direction lies in bridging the IB principle with large-scale multimodal foundation models. Although these models are not derived from IB formulations, their representation learning objectives inherently aim to learn compact latent embeddings that preserve task-relevant semantics while suppressing redundant modality-specific variations. Contrastive alignment, masked modeling, and cross-modal fusion strategies can be interpreted as practical approximations of the Variational Information Bottleneck, encouraging minimal yet sufficient representations for downstream tasks. Future ToSC systems may systematically integrate IB-guided regularization into multimodal foundation models to achieve adaptive semantic compression, improved generalization across tasks, and communication-efficient deployment in AI-native wireless networks. Another important direction involves integrating unequal error protection (UEP) into semantic transmission frameworks. Rather than uniformly protecting all transmitted symbols, task-critical features can be prioritized through adaptive power allocation or coding redundancy, improving task reliability under constrained bandwidth. In multi-user and multi-task scenarios, joint adaptation must coordinate semantic compression with cross-layer resource allocation. Reinforcement learning-based schedulers have been proposed to dynamically allocate power, bandwidth, and transmission rates according to semantic importance and long-term task objectives. Despite these advances, several open challenges remain. Accurately modeling the coupling between semantic distortion and physical-layer impairments remains theoretically underdeveloped. Similarly, achieving stable adaptive training under highly dynamic channels and scaling adaptive JSCC to multimodal foundation models introduce significant computational and synchronization overhead. Addressing these issues is essential for deploying ToSC systems in realistic, large-scale wireless environments. In addition, an emerging frontier for ToSC lies in supporting embodied AI agents, such as autonomous vehicles, UAV swarms, collaborative robots, and edge-intelligent IoT systems. The conventional perception-only applications, embodied agents operate in closed-loop environments where sensing, communication, decision-making, and actuation are tightly coupled. In such systems, communication is no longer a passive information delivery process but an integral component of real-time control and cooperative reasoning. From a ToSC perspective, semantic transmission for embodied agents must prioritize control-relevant state representations rather than raw sensory data. For example, agents may share compact latent embeddings describing object trajectories, obstacle states, or intent predictions. This enables low-latency coordination while preserving task semantics under bandwidth constraints. Also, multi-agent embodied systems introduce additional challenges, including distributed perception fusion, cooperative policy learning, and dynamic task allocation. Semantic communication frameworks must jointly consider latency sensitivity, reliability, and control stability. Theoretical analysis of the impact of semantic distortion on closed-loop control performance remains largely unexplored. Additionally, integrating foundation models into embodied semantic communication raises new challenges in model compression, real-time inference, and synchronization across heterogeneous agents. Future work should therefore investigate control-aware semantic metrics, stability-guaranteed semantic adaptation, and cross-layer co-design frameworks that unify perception, communication, and action. Developing standardized benchmarks for embodied ToSC systems will also be critical to evaluating robustness, scalability, and generalization in realistic dynamic environments.

5. Conclusions

This article has provided a comprehensive review of ToSC, encompassing its theoretical foundations, system architectures, enabling technologies, and representative applications. We discussed the principles of task-relevant information extraction, semantic encoding and compression, joint source–channel processing, and adaptive transmission strategies. A structured analysis of recent developments was presented, covering both single-agent and multi-agent scenarios, along with techniques for robustness enhancement under dynamic wireless conditions. In addition to consolidating current research, this survey examined the interplay between semantic processing and emerging communication infrastructures, including edge–cloud collaboration, intelligent reflecting surfaces, and AI-native networking. Our analysis of these technologies’ requirements across a variety of application domains, from autonomous control to smart manufacturing, allowed us to identify important performance trade-offs in latency, accuracy, and bandwidth efficiency. Co-design across the communication, computing, and learning layers are important for the future development of ToSC. Multimodal semantic representation integration, distributed agent coordination, and dynamic adaptability to task demands and channel conditions present research opportunities. With LLM-powered semantic extraction and reasoning, we expect that effective and reliable communication between embodied agents will become possible for embodied AI agents. For example, Multimodal LLMs capable of jointly reasoning over text, vision, audio, and structured data provide powerful pretrained semantic encoders that can extract highly structured, context-aware latent representations. Instead of transmitting raw signals or extracted feature embeddings, future ToSC systems may communicate compressed reasoning states, intent representations, or task-conditioned latent tokens generated by foundation models. Moreover, the generative capabilities of LLMs enable semantic reconstruction and inference even under noisy information, improving robustness in constrained wireless environments. Integrating LLM-driven reasoning with adaptive joint source–channel coding, cross-layer resource optimization, and multi-agent coordination may lead to AI-native communication systems.