On-Device and Cloud-Based Learning for Next-Generation AI Applications

1. Introduction

In recent years, the rapid advancement of deep learning has brought about transformative changes across a wide spectrum of application domains, ranging from computer vision, natural language processing, and speech recognition to autonomous driving, healthcare, and smart Internet-of-Things (IoT) environments. Traditionally, state-of-the-art deep learning models have required immense computational resources and large-scale datasets for both training and inference. These models, typically comprising millions or even billions of parameters, such as BERT, GPT, and ResNet, are predominantly trained and deployed in centralized cloud data centers where abundant computational resources and storage capabilities are available. However, the proliferation of mobile devices, edge computing infrastructures, and embedded systems has introduced a new paradigm in deep learning: the desire and necessity to perform inference, and to some extent training, on resource-constrained devices at the edge of the network [1]. This shift is driven by multiple compelling factors [2]. First, there is an increasing demand for real-time, low-latency inference in applications such as augmented reality, robotics, and smart homes, which cannot tolerate the round-trip latency incurred by transmitting data to remote cloud servers [3]. Second, concerns regarding data privacy, security, and compliance with regulations such as GDPR have motivated on-device processing to minimize the exposure of sensitive user data. Third, the sheer volume of data generated at the edge makes it impractical and cost-prohibitive to transmit all data to the cloud for processing, thus necessitating more intelligent and distributed computing paradigms [4]. Nonetheless, the limited computational power, energy constraints, and memory limitations of edge devices pose significant challenges to the deployment of large-scale deep learning models directly on-device. To address these constraints, various techniques such as model quantization, pruning, knowledge distillation, and architecture search for efficient models (e.g., MobileNet, TinyBERT) have been proposed to enable the design and deployment of small deep learning models that can operate within the capabilities of edge hardware [5]. Despite these advances, small models often suffer from reduced accuracy and generalization performance compared to their large-scale cloud counterparts, thereby creating a trade-off between efficiency and performance [6]. This confluence of cloud and edge computing capabilities has given rise to the concept of collaborative learning, a hybrid learning paradigm where small models on edge devices and large models in the cloud cooperate to optimize inference and learning tasks. Collaborative learning seeks to harness the complementary strengths of both edge and cloud: the proximity, privacy, and responsiveness of on-device models, and the accuracy, computational richness, and data aggregation abilities of cloud-based models. The overarching objective is to design synergistic frameworks that allow seamless interaction between these heterogeneous models while addressing challenges related to communication overhead, model consistency, system heterogeneity, and dynamic environmental conditions [7]. Several collaborative learning frameworks have been proposed, including but not limited to, split computing (also known as split learning), where a deep neural network is partitioned between the edge and cloud; federated learning, where decentralized models are trained collaboratively without sharing raw data; and knowledge distillation-based approaches, where large cloud models distill knowledge to small edge models either offline or dynamically during inference. Moreover, hybrid techniques that integrate multiple paradigms, such as federated distillation or split-federated learning, have emerged to exploit the advantages of each method. Beyond the methodological developments, the deployment of collaborative learning systems introduces a range of system-level challenges, including network variability, energy management, model synchronization, data heterogeneity, and security threats such as model inversion and poisoning attacks. Consequently, the design of robust, efficient, and secure collaborative learning architectures demands a cross-disciplinary approach, integrating insights from machine learning, systems engineering, communication networks, and security [8]. This survey provides a comprehensive and structured review of the emerging field of collaborative learning between on-device small models and cloud-based large deep learning models. Our aim is to elucidate the fundamental principles, technical challenges, and current state-of-the-art methodologies in this domain [9]. We categorize and analyze existing approaches, highlight key design trade-offs, and identify promising directions for future research. Specifically, we address the following aspects in this survey:

• The motivation and rationale for collaborative learning, with an emphasis on application scenarios and system constraints [10].
• A taxonomy of collaborative learning frameworks, including split learning, federated learning, and knowledge distillation, and their adaptations for heterogeneous model collaboration.
• Technical challenges inherent to collaborative learning, such as latency, energy consumption, model heterogeneity, and security, along with potential mitigation strategies.
• An overview of real-world systems and platforms that support collaborative learning, including hardware, software, and network considerations.
• Open research questions and future directions, including the role of foundation models, personalized learning, adaptive collaboration strategies, and standardization efforts [11].

In summary, as the demand for intelligent edge applications continues to grow, and the limitations of both standalone cloud and edge computing become more pronounced, collaborative learning offers a promising pathway towards scalable, efficient, and privacy-preserving deep learning systems. This survey seeks to equip researchers, practitioners, and system designers with a deep understanding of the landscape of collaborative learning and to foster the development of next-generation intelligent systems that seamlessly blend the capabilities of on-device small models with those of cloud-based large models [12].

2. Problem Formulation

Let us formally define the problem of collaborative learning between an on-device small model and a cloud-based large deep learning model. We denote the input space by $\mathcal{X}\subseteq {\mathbb{R}}^d$ and the corresponding output or label space by $\mathcal{Y}$. Given a data distribution $\mathcal{D}$ over $\mathcal{X}\times \mathcal{Y}$, the goal of a learning system is to find a function $f:\mathcal{X}\to \mathcal{Y}$ that minimizes a loss function $\mathcal{L}\left(f\right(x),y)$, where $(x,y)\sim \mathcal{D}$. In the context of collaborative learning, this function f is not implemented by a single monolithic model, but rather decomposed into two (or more) components that are distributed across the edge device and the cloud infrastructure [13]. Let us denote the on-device small model by $f_s:\mathcal{X}\to \mathcal{Z}$, where $\mathcal{Z}\subseteq {\mathbb{R}}^k$ represents an intermediate feature space, and the cloud-based large model by $f_l:\mathcal{Z}\to \mathcal{Y}$ [14]. The complete model is then given by the composition $f\left(x\right)=f_l\left(f_s\left(x\right)\right)$. The optimization objective is to minimize the expected loss:

$$\underset{f_s,f_l}{min}{\mathbb{E}}_{(x,y)\sim \mathcal{D}}\left[\mathcal{L}(f_l\left(f_s\left(x\right)\right),y)\right].$$

(1)

In practice, only a finite dataset ${\left\{(x_i,y_i)\right\}}_{i=1}^n$ sampled from $\mathcal{D}$ is available. Moreover, this dataset may be distributed across edge devices and cloud servers, denoted by ${\mathcal{D}}_e$ and ${\mathcal{D}}_c$ respectively. In some cases, data cannot be directly shared between devices and the cloud due to privacy constraints, leading to the need for distributed optimization strategies [15]. Let us denote the empirical loss over the dataset by:

$$\widehat{\mathcal{L}}(f_s,f_l)={\displaystyle \frac{1}{n}}\sum _{i=1}^n\mathcal{L}(f_l\left(f_s\left(x_i\right)\right),y_i).$$

(2)

The problem then becomes minimizing $\widehat{\mathcal{L}}$ subject to system constraints:

$$\begin{array}{cc}\hfill \underset{f_s,f_l}{min}& \widehat{\mathcal{L}}(f_s,f_l)\hfill \\ \hfill \mathrm{subject}\mathrm{to}& C(f_s,f_l)\le \tau ,\hfill \end{array}$$

(3)

where $C(f_s,f_l)$ represents a composite cost function encompassing computation time, energy consumption, communication latency, and privacy leakage, and $\tau$ is a user-defined threshold. One of the primary challenges in collaborative learning lies in the optimal partitioning of the model and the coordination between $f_s$ and $f_l$. For instance, in split learning, the partition point $\pi$ determines the layer at which the model is split between the edge and the cloud [16]. Let $f(x;\pi )=f_l^{[\pi +1:L]}\left(f_s^{[1:\pi ]}\left(x\right)\right)$, where L denotes the total number of layers in the complete model. Finding an optimal partition point requires balancing the computational cost $C_e\left(\pi \right)$ on the edge and $C_c\left(\pi \right)$ on the cloud, as well as the communication cost $C_{comm}\left(\pi \right)$ incurred in transmitting the intermediate activation $z=f_s^{[1:\pi ]}\left(x\right)$ from the edge to the cloud. Therefore, the partition optimization problem can be formalized as:

$$\underset{\pi }{min}{C}_e\left(\pi \right)+C_c\left(\pi \right)+C_{comm}\left(\pi \right),\mathrm{s}.\mathrm{t}.\mathcal{L}(f(x;\pi ),y)\le ϵ,$$

(4)

where $ϵ$ denotes an acceptable loss tolerance. In knowledge distillation-based collaborative learning, the objective shifts towards minimizing a distillation loss ${\mathcal{L}}_{KD}$ between the output of the small model $f_s$ and a large teacher model $f_T$ deployed in the cloud [17]. The distillation objective is typically formulated as:

$${\mathcal{L}}_{KD}=\alpha ·{\mathcal{L}}_{CE}(f_s\left(x\right),y)+(1-\alpha )·{\mathcal{L}}_{KL}(f_s\left(x\right),f_T\left(x\right)),$$

(5)

where ${\mathcal{L}}_{CE}$ denotes the cross-entropy loss, ${\mathcal{L}}_{KL}$ is the Kullback–Leibler divergence between the soft outputs of $f_s$ and $f_T$, and $\alpha \in [0,1]$ is a hyperparameter balancing the two terms. The goal is to train the on-device small model to mimic the predictions of the large cloud model while retaining acceptable computational efficiency. In federated learning scenarios, let us consider a set of M devices, each with a local dataset ${\mathcal{D}}_m$, $m=1,\dots ,M$ [18]. The collaborative objective is to minimize the global loss:

$$\underset{f_s}{min}\sum _{m=1}^M{\displaystyle \frac{|{\mathcal{D}}_m|}{n}}{\widehat{\mathcal{L}}}_m\left(f_s\right),$$

(6)

where ${\widehat{\mathcal{L}}}_m\left(f_s\right)={\displaystyle \frac{1}{|{\mathcal{D}}_m|}}{\sum }_{(x_i,y_i)\in {\mathcal{D}}_m}\mathcal{L}(f_s\left(x_i\right),y_i)$. The large cloud model $f_l$ may serve as an aggregator or teacher, distilling knowledge to the federated edge models via a server-based coordination protocol [19]. In summary, the mathematical formulation of collaborative learning encompasses diverse optimization objectives and constraints, dependent on the chosen collaboration paradigm—split learning, federated learning, knowledge distillation, or hybrid models [20]. Each paradigm introduces unique computational and communication trade-offs, necessitating multi-objective optimization approaches to balance performance, efficiency, and privacy [21]. Additionally, dynamic environmental factors such as network bandwidth fluctuations, device heterogeneity, and user mobility introduce further complexity into this optimization landscape, making adaptive and robust solutions critical to real-world deployment. This section sets the mathematical foundation upon which subsequent discussions of specific methodologies and systems will be built [22].

3. System Architecture

A fundamental aspect of collaborative learning between on-device small models and cloud-based large models lies in the design and implementation of an efficient, scalable, and adaptable system architecture [23]. This architecture must orchestrate the interactions between edge devices and cloud servers to facilitate joint model inference and training while satisfying constraints related to latency, privacy, energy, and communication bandwidth [24]. Figure 1 presents a vertical overview of a typical collaborative learning architecture [25]. At the bottom layer are Edge Devices, which include smartphones, IoT sensors, autonomous vehicles, and wearable devices. Each device contains limited computational resources—CPU, GPU, or specialized AI accelerators—and stores a lightweight model component $f_s$, along with a local dataset ${\mathcal{D}}_e$ [26]. These devices perform initial data processing and model inference, and optionally contribute to local training [27]. Above the edge lies the Communication Layer, responsible for managing data transmission between edge and cloud. This layer accounts for variable network conditions, latency constraints, and security protocols such as encryption and differential privacy. It also supports compression and quantization of intermediate representations to minimize communication overhead. At the top is the Cloud Infrastructure, which hosts the large deep learning model $f_l$, trained on massive datasets ${\mathcal{D}}_c$. The cloud has high computational capacity, enabling intensive tasks such as fine-tuning, centralized aggregation, and large-scale inference [28]. It also coordinates collaborative protocols, maintains global model versions, and performs knowledge distillation to enhance the performance of edge models.

The edge-cloud collaboration depicted in Figure 1 supports various operational modes [29]. In inference mode, raw input x is first processed by the small model $f_s$ on the device to generate intermediate features $z=f_s\left(x\right)$ [30]. These features are transmitted through the communication layer to the cloud, where the large model $f_l$ completes the inference, producing $\widehat{y}=f_l\left(z\right)$ [31]. This division reduces latency while leveraging the accuracy of the large model [32]. In training mode, either federated learning or knowledge distillation may be used. In federated learning, each edge device trains $f_s$ on local data and periodically sends model updates (e.g., gradients or weights) to the cloud, which aggregates them into a global model. In distillation, the cloud model $f_l$ sends distilled outputs or soft labels to the edge, enhancing the training of $f_s$ without exposing raw data. A critical component of this architecture is the communication protocol [33]. Due to bandwidth constraints and energy limitations on edge devices, it is essential to compress the transmitted data z without significantly degrading model performance [34]. Techniques such as sparsification, quantization, and entropy coding are commonly used [35]. Additionally, privacy-preserving mechanisms such as homomorphic encryption, secure multi-party computation, or differential privacy can be applied to z to protect sensitive user information [36]. Furthermore, the architecture must adapt to heterogeneous edge devices with varying capabilities [37]. Dynamic model partitioning allows the system to select the optimal split point $\pi$ in the model, based on real-time monitoring of network latency, energy availability, and computational load. This requires a runtime decision engine, potentially driven by reinforcement learning, to balance the trade-offs between local computation and remote processing [38]. In conclusion, the system architecture for collaborative learning must holistically integrate edge computation, communication efficiency, and cloud intelligence. The interplay between small and large models across this architecture enables a continuum of intelligence from edge to cloud, empowering applications that demand responsiveness, privacy, and high accuracy simultaneously [39]. Subsequent sections will delve into the specific methodologies that instantiate this architecture in practical systems.

4. Comparison of Collaborative Learning Paradigms

Collaborative learning between on-device small models and cloud-based large models can be implemented through several distinct paradigms, each with its own operational principles, advantages, and limitations [40]. Among the most widely adopted frameworks are Split Learning, Federated Learning, and Knowledge Distillation, along with emerging hybrid models that combine elements from these approaches [41]. A systematic comparison of these paradigms is crucial for understanding the design trade-offs involved and for selecting the most appropriate strategy for a given application scenario. Table 1 provides a comparative summary of the key characteristics of the major collaborative learning paradigms. The comparison covers aspects such as the structure of model partitioning, data locality, communication patterns, privacy implications, computational load distribution, and typical use cases [42]. Each paradigm is tailored to address specific constraints and goals, and their suitability varies depending on the context, such as the heterogeneity of edge devices, network conditions, and privacy requirements.

Split Learning offers a fine-grained division of computation between edge and cloud, making it particularly suitable for real-time inference in latency-sensitive applications [43]. In this paradigm, the edge device processes the input x through the early layers of the model, generating intermediate activations z, which are sent to the cloud. The cloud completes the inference or backpropagation and may return gradients or output predictions [44]. While this approach minimizes data exposure and supports dynamic model partitioning, it requires low-latency, high-bandwidth connections and can be communication-intensive during training. Federated Learning decentralizes the training process by distributing full copies of a small model to each participating edge device. Devices perform local training on their own data and periodically transmit model updates, such as weight deltas or gradients, to a central server [45]. The server aggregates these updates to produce a global model, which is redistributed [46]. This method excels in privacy preservation and supports massive scalability, but imposes significant computation on edge devices and may struggle with data heterogeneity (non-IID data) and stragglers in asynchronous environments. Knowledge Distillation enables indirect collaboration by having a powerful cloud model (teacher) generate softened labels or feature representations, which are then used to train a smaller model (student) on the edge. This can occur either offline or online, and may involve continual distillation as the student model encounters new data. This approach is flexible in terms of communication frequency and can significantly compress large models for edge deployment [47]. However, its effectiveness depends on the alignment between teacher and student models and may require repeated cloud access for optimal performance [48]. Ultimately, the choice of paradigm depends on the specific application requirements, such as the need for privacy, latency constraints, and the computational capacity of edge devices. Hybrid models, such as split-federated learning or federated distillation, are gaining popularity as they attempt to combine the strengths of different approaches [49]. For example, federated learning may be augmented with distillation to reduce communication overhead, or split learning may be integrated with privacy-preserving techniques to enhance security. In conclusion, understanding the comparative landscape of collaborative learning paradigms allows system designers and researchers to better align technical capabilities with application demands [50]. As edge-cloud ecosystems become more diverse and pervasive, the ability to flexibly adopt and combine different paradigms will be crucial for building robust, efficient, and intelligent systems.

5. Challenges and Research Opportunities

While collaborative learning between on-device small models and cloud-based large models presents immense potential, realizing its full benefits requires overcoming several fundamental and practical challenges. These challenges span multiple dimensions, including system-level constraints, algorithmic limitations, privacy-preserving mechanisms, and real-world deployment concerns. Addressing these issues not only enables more robust and efficient systems but also opens up fertile ground for impactful research and innovation. One of the foremost challenges is the optimization of communication efficiency between edge and cloud [51]. As collaborative learning often involves frequent exchange of intermediate data, model parameters, or gradients, the communication cost can become a bottleneck, especially in bandwidth-constrained or latency-sensitive environments [52]. In split learning, for instance, transmitting high-dimensional feature maps z at each forward pass imposes significant load on the network, which may not be sustainable for real-time applications or in regions with limited connectivity. Advanced techniques such as adaptive compression, sparse representation learning, and progressive transmission of features are actively being explored to mitigate this issue. Additionally, dynamically selecting the model partition point $\pi$ based on current network conditions and computational load is a promising strategy, but it requires sophisticated orchestration mechanisms and real-time monitoring. Another critical issue is data heterogeneity and non-IID distribution across edge devices [53]. In federated and distillation-based learning paradigms, each device typically collects data unique to its environment and user behavior, leading to statistically diverse local datasets ${\mathcal{D}}_e^{\left(i\right)}$ [54]. This non-IID nature of data introduces challenges in achieving convergence and generalization, as model updates from different devices may be conflicting or biased. Consequently, aggregation methods such as weighted averaging in federated learning (e.g., FedAvg) may lead to suboptimal performance [55]. Research is increasingly focusing on personalized federated learning, meta-learning, and domain adaptation techniques that can account for data heterogeneity and tailor the global model to diverse local conditions without compromising overall performance. Privacy and security concerns are also paramount in collaborative learning scenarios. Although raw data typically remains on the edge, intermediate features or model updates may still leak sensitive information. For example, in split learning, it has been shown that activations z can be exploited through inversion attacks to reconstruct input data. Similarly, in federated learning, gradients may reveal private data if not properly sanitized [56]. Techniques such as differential privacy, secure multi-party computation (SMPC), homomorphic encryption, and trusted execution environments (TEE) offer potential solutions, but each comes with trade-offs in terms of computational cost, latency, and scalability [57]. A key research direction involves developing lightweight, scalable privacy-preserving mechanisms that can be deployed efficiently across heterogeneous devices with limited resources. From an algorithmic perspective, model co-design poses a unique challenge [58]. Collaborative learning necessitates designing models that are not only accurate but also modular, with clearly defined interfaces between the small model $f_s$ and large model $f_l$ [59]. This requires innovations in neural network architectures that support efficient partitioning, distillation, or transfer of knowledge [60]. Techniques such as neural architecture search (NAS), modular neural networks, and efficient transformer variants (e.g., MobileBERT, TinyViT) are being adapted to facilitate this co-design [61]. Moreover, dynamically reconfigurable models that adjust their complexity or depth based on available resources can offer additional flexibility, but they require reliable runtime control mechanisms and robust performance guarantees. A further challenge lies in the evaluation and benchmarking of collaborative learning systems. Traditional metrics such as accuracy and inference time do not fully capture the multi-dimensional trade-offs involved, including energy consumption, privacy risk, communication overhead, and system robustness. Standardized benchmarks, datasets, and simulation environments that reflect realistic edge-cloud scenarios are urgently needed. Such benchmarks should account for diverse device capabilities, network conditions, and user behaviors to enable fair comparison and reproducibility of research outcomes. Finally, scalability and deployment at scale remain significant hurdles. Real-world applications involve a large number of edge devices, each with varying hardware, software, and network configurations [62]. Efficient orchestration, fault tolerance, and update propagation in such heterogeneous environments are complex yet essential for practical adoption. Emerging technologies such as 5G, edge AI accelerators, and distributed computing frameworks (e.g., Kubernetes, TensorFlow Federated) offer infrastructure support, but their integration into collaborative learning workflows remains an open research area. In conclusion, while collaborative learning between on-device small models and cloud-based large models holds transformative potential, realizing this vision necessitates overcoming substantial challenges across communication, computation, privacy, and scalability [63]. These challenges, however, simultaneously represent rich research opportunities. Future work that integrates advances in machine learning, systems engineering, and privacy-enhancing technologies will be crucial in making collaborative learning not only feasible but also ubiquitous across diverse domains such as healthcare, autonomous systems, smart cities, and personal AI.

6. Applications and Case Studies

The collaborative learning paradigm—where on-device small models work in tandem with powerful cloud-based deep learning models—has begun to transform a wide range of application domains. This transformation is driven by the growing need for systems that balance the often conflicting requirements of responsiveness, accuracy, privacy, and energy efficiency [64]. In this section, we explore several prominent application areas and case studies that demonstrate the tangible benefits and practical considerations of deploying collaborative learning systems in real-world environments [65]. One of the most impactful areas is healthcare and personalized medicine, where patient data is highly sensitive and privacy regulations such as HIPAA and GDPR impose strict data protection requirements. In scenarios such as wearable health monitoring or mobile diagnostics, edge devices such as smartwatches or smartphones continuously collect physiological data (e.g., heart rate, blood oxygen levels, ECG signals). On-device models provide immediate feedback, detecting anomalies or triggering alerts in real-time. However, due to the limited capacity of edge models, complex diagnostic tasks or longitudinal pattern recognition may necessitate cloud-based analysis. Through collaborative learning—often via split learning or federated learning—the edge can process raw signals locally and send compact, privacy-preserving representations to the cloud, which refines the diagnosis using advanced models trained on large-scale medical datasets. Case studies have demonstrated this approach in early detection of arrhythmias, diabetic retinopathy screening, and even mental health monitoring via speech and behavioral analysis, offering both high accuracy and robust privacy [66]. In the realm of autonomous vehicles and intelligent transportation systems, real-time decision-making is critical, yet full reliance on cloud services is infeasible due to latency and connectivity limitations. Autonomous vehicles must process sensor data (e.g., LiDAR, camera, radar) rapidly to perform tasks such as object detection, path planning, and collision avoidance. On-board models offer low-latency inference but are constrained in complexity due to energy and space limitations [67]. Collaborative learning enables these systems to offload computationally intensive tasks, such as complex scene understanding or global route optimization, to the cloud [68]. For example, a vehicle might detect objects locally but send semantically rich representations to the cloud for high-level decision support, receiving back optimized navigation strategies. Furthermore, federated learning can be used to aggregate driving experiences from multiple vehicles without exposing raw sensor data, leading to continual improvement of both edge and cloud models [69]. These systems also benefit from real-time cloud updates, enabling vehicles to adapt quickly to new traffic patterns or hazardous conditions [70]. Smart cities and IoT infrastructure represent another fertile domain for collaborative learning [71]. In such environments, a multitude of distributed sensors and edge devices collect data related to energy consumption, air quality, traffic flow, and public safety. These devices, typically constrained in power and compute, require efficient local inference to enable immediate responses, such as triggering alarms or adjusting environmental controls [72]. However, the aggregate data across a city provides valuable insights for long-term planning and policy-making. Collaborative learning enables both immediate local responsiveness and comprehensive global analysis. For instance, in energy management systems, local devices predict energy demand and adjust consumption autonomously, while cloud-based models forecast city-wide energy trends and optimize grid operations [73]. Split learning facilitates secure, real-time interaction between local and centralized intelligence, while federated learning ensures that sensitive user data remains on-premise. In the field of natural language processing (NLP) and personal AI assistants, collaborative learning is crucial for enabling personalized yet privacy-preserving interactions. Virtual assistants like Siri, Alexa, or Google Assistant must respond quickly to voice commands, often without reliable network connectivity [74]. On-device models handle basic commands and wake-word detection, while cloud-based large language models provide more sophisticated understanding and dialog generation [75]. Through distillation and collaborative inference, the on-device models can continually improve from cloud-based models without exposing user conversations [76]. Moreover, federated learning allows models to learn from user interactions across millions of devices while ensuring that individual voice data never leaves the device. Recent studies have shown that such approaches can significantly improve the quality and personalization of dialog systems while maintaining strong privacy guarantees. Finally, in augmented reality (AR) and mobile gaming, collaborative learning enables rich, immersive experiences on resource-constrained devices. AR applications require real-time processing of video, depth, and spatial data to render virtual content accurately and responsively. On-device models perform rapid pose estimation and environment mapping, while cloud models handle complex tasks such as object recognition, scene understanding, or multiplayer coordination [77]. Collaborative inference ensures that latency-critical operations are kept local, while cloud intelligence enhances realism and consistency across devices [78]. Case studies in mobile AR gaming and industrial AR support systems highlight how hybrid model deployment enables scalable, low-latency experiences that were previously only possible on high-end hardware [79]. In summary, collaborative learning between on-device small models and cloud-based large models is being actively deployed across diverse application domains, each with unique requirements and challenges. The key to successful deployment lies in carefully balancing the division of labor between edge and cloud, optimizing communication, and ensuring privacy and robustness. As hardware capabilities continue to evolve and networks become more pervasive, the reach and impact of collaborative learning systems are poised to expand dramatically, enabling smarter, more responsive, and more privacy-respecting technologies across all facets of daily life.

7. Future Directions and Emerging Trends

As collaborative learning continues to gain traction in both academic research and industrial deployment, several emerging trends and future directions are shaping its evolution [80]. These trends are influenced not only by advances in machine learning algorithms but also by innovations in hardware, networking, privacy technologies, and system-level design. The future of collaborative learning will be characterized by increasing intelligence and autonomy at the edge, more seamless and efficient integration with the cloud, and greater personalization, security, and adaptability of AI systems across diverse application contexts. One significant future direction lies in the development of adaptive and dynamic collaboration mechanisms [81]. Current collaborative learning systems often rely on static configurations, where the division of computation and learning responsibilities between edge and cloud is fixed or predetermined. However, real-world environments are inherently dynamic, with fluctuating network bandwidth, varying computational resources, and evolving user requirements [82]. Future systems must be capable of dynamically adjusting the model partitioning point $\pi$, selectively offloading tasks based on current resource availability, latency constraints, or privacy concerns. Such systems require real-time monitoring, predictive analytics for resource forecasting, and reinforcement learning-based strategies that can learn optimal collaboration policies over time. Additionally, dynamic model reconfiguration, such as elastic neural networks that adjust their size or depth on-the-fly, will play a key role in enabling fluid, context-aware collaborative intelligence [83]. Another critical area of growth is the integration of privacy-enhancing technologies (PETs) into collaborative learning workflows [84]. While methods such as differential privacy, homomorphic encryption, and secure multi-party computation have seen increasing adoption, future systems will need to embed these technologies more deeply and efficiently into learning pipelines [85]. Lightweight cryptographic protocols, efficient privacy budget management, and user-controllable privacy settings will become essential, particularly in consumer-facing applications such as personal AI assistants and health monitoring [86]. Moreover, the intersection of PETs with federated and split learning opens up new possibilities, such as privacy-preserving split learning or federated analytics, where both raw data and intermediate representations remain secure. The design of PET-aware model architectures, communication protocols, and optimization algorithms is an emerging field that will significantly influence the deployment of trustworthy AI systems [87]. The rise of specialized edge AI hardware represents another transformative trend. Dedicated AI accelerators, such as Tensor Processing Units (TPUs), Neural Processing Units (NPUs), and custom ASICs, are becoming increasingly prevalent in smartphones, IoT devices, and embedded systems. These accelerators enable low-latency, energy-efficient execution of inference and training tasks on the edge, thus expanding the scope of collaborative learning. Future research will focus on co-designing algorithms and hardware to maximize performance within stringent power and thermal envelopes. Techniques such as quantization-aware training, hardware-aware neural architecture search (HW-NAS), and compiler optimizations for AI inference (e.g., TVM, XLA) will be critical enablers of efficient collaboration [88]. Additionally, the emergence of neuromorphic computing and in-memory processing holds promise for ultra-low-power edge intelligence, further enhancing the feasibility of real-time, on-device learning [89]. Another promising direction is the advancement of continual and lifelong learning in collaborative settings. In many real-world scenarios, data is not static but evolves over time, reflecting changes in user behavior, environment, and system requirements [90]. Collaborative learning systems must be capable of incremental learning, continually adapting to new data without catastrophic forgetting [91]. This requires the development of algorithms that support on-device continual learning, coupled with cloud-assisted memory consolidation and knowledge transfer [92]. Techniques such as elastic weight consolidation, experience replay, and meta-learning can be adapted to collaborative settings, enabling personalized and adaptive AI experiences that improve over time [93]. Moreover, decentralized learning paradigms, where learning occurs in peer-to-peer networks without centralized cloud orchestration, may emerge as viable alternatives in privacy-critical or resource-constrained environments. Finally, the future of collaborative learning will be shaped by the emergence of standardized frameworks, benchmarks, and platforms that facilitate experimentation, evaluation, and deployment. Current research is often siloed, with limited comparability due to the lack of common datasets, metrics, or system configurations [94]. Open-source platforms such as TensorFlow Federated, PySyft, and Flower are beginning to address this gap, but there is a pressing need for comprehensive toolchains that integrate model design, privacy controls, communication optimization, and deployment orchestration. Benchmarks that reflect realistic edge-cloud scenarios—including heterogeneous devices, dynamic network conditions, and adversarial threats—will enable more robust evaluation of proposed methods [95]. Furthermore, the development of simulation environments and digital twins for collaborative learning will facilitate rapid prototyping and stress-testing of systems under diverse conditions [96]. In conclusion, the future of collaborative learning lies at the intersection of machine learning, systems engineering, privacy science, and hardware design. The trajectory of research and development will be shaped by the need for intelligent, secure, and adaptable systems that seamlessly integrate the strengths of edge and cloud. As collaborative learning matures, it has the potential to unlock transformative capabilities across sectors, from personalized healthcare and autonomous systems to smart environments and beyond [97]. Continued interdisciplinary collaboration and innovation will be essential to realize this vision and to address the profound technical and societal challenges it presents.

8. Conclusions

The proliferation of intelligent devices and the exponential growth of deep learning capabilities have converged to create both opportunities and challenges in the deployment of artificial intelligence across diverse environments. At the heart of this convergence lies the paradigm of collaborative learning between on-device small models and cloud-based large models—a paradigm that seeks to combine the immediacy, privacy, and autonomy of edge computing with the power, scalability, and holistic intelligence of the cloud [98]. This survey has examined the multifaceted aspects of this collaborative learning landscape, encompassing fundamental methodologies, technical challenges, practical applications, and emerging research directions [99]. One of the core themes that emerges from this survey is the intricate balance that collaborative learning seeks to strike among competing system objectives. On-device models are constrained by limited compute, storage, and energy resources, yet they offer unique advantages such as low-latency inference, enhanced privacy, and real-time responsiveness [100]. In contrast, cloud-based models leverage vast computational resources and massive datasets, enabling sophisticated analysis, continual model updates, and global context-awareness [101]. Collaborative learning frameworks—including split learning, federated learning, knowledge distillation, and hybrid approaches—provide flexible mechanisms to distribute computational and learning workloads across the edge-cloud continuum, dynamically adapting to system constraints and user needs. The design and deployment of such systems, however, is far from trivial [102]. Communication efficiency, privacy preservation, robustness to heterogeneous environments, and the ability to personalize models without compromising scalability are persistent challenges [103]. The interplay between model design and system architecture is particularly critical—requiring co-optimization of algorithms, network protocols, and hardware acceleration [104]. The survey has outlined how recent advances in edge AI hardware, privacy-enhancing technologies, and dynamic model orchestration are beginning to address these challenges, but significant research and engineering work remains. Moreover, the applications of collaborative learning are rapidly expanding across domains [105]. From real-time health monitoring and autonomous driving to smart infrastructure and natural language interfaces, collaborative learning systems are enabling intelligent functionalities that were previously unattainable in resource-constrained settings. These applications not only demonstrate the practical feasibility of collaborative learning but also highlight its societal impact—in improving accessibility, enhancing safety, protecting privacy, and delivering personalized services at scale. Looking forward, the future of collaborative learning promises even greater integration and intelligence. Emerging trends such as adaptive learning architectures, privacy-centric AI governance, continual and lifelong learning, and standardized benchmarking will drive the next wave of innovation. Collaborative learning is poised to become a foundational element in the architecture of ubiquitous AI—permeating every aspect of human-computer interaction, embedded intelligence, and distributed computing [106].

In conclusion, the collaborative learning of on-device small models and cloud-based large models represents a transformative shift in the design of intelligent systems. It transcends traditional boundaries between centralized and decentralized computing, offering a unified approach that leverages the best of both worlds. Realizing the full potential of this paradigm will require sustained, interdisciplinary efforts that bridge machine learning, systems engineering, privacy science, and user-centric design. As these efforts mature, collaborative learning will not only enhance the capabilities of AI systems but also ensure that these capabilities are delivered in ways that are efficient, equitable, secure, and aligned with human values.