A Standardized On-Chip Communication Interface for Dynamic Hardware/Software Task Migration in Heterogeneous SoPCs

1. Introduction

A heterogeneous system-on-programmable-chip (SoPC) can execute control-oriented tasks on a processor and parallel kernels in field-programmable gate array (FPGA) fabric. Moving a task between these domains is attractive when workload criticality or resource availability changes. However, placement flexibility is useful only when communication, task state, and in-flight data remain consistent. Reconfigurable-computing operating systems have long addressed parts of this integration problem, including hardware execution abstractions and access to operating-system services [1].

The problem considered here is the interface visible to communicating tasks. A producer should address a logical consumer rather than encode whether that consumer is currently a software thread or a hardware module. Similarly, replacement of a hardware module should not expose residual mailbox contents to the wrong consumer. A common API alone does not resolve these issues: route updates, completion semantics, state serialization, and memory ownership must agree at the migration boundary.

This paper develops the Int-S/Int-H architecture around a software manager, a hardware manager, local mailboxes, and a static routing fabric. The term standardized denotes a uniform contract within this architecture; it does not imply adoption by a standards organization or interoperability with arbitrary third-party IP. Hardware and software implementations must explicitly conform to the same task-state and message representation.

The contributions are threefold. First, the task-facing contract and manager responsibilities are specified independently of peer placement. Second, migration is decomposed into distinct consistency and timing obligations, including DMA-visible state and cache ownership. Third, a reproducible closed-form analysis separates payload transport, mailbox preservation, and partial-bitstream transfer. The study does not report synthesis, place-and-route, FPGA-board measurements, or a formal verification result. Its scope is an architectural proposal supported by transparent calculations and a testable implementation plan.

2. Related Work and Positioning

LEAP provides latency-insensitive communication and platform services intended to improve FPGA application portability [2]. ReconOS presents hardware and software threads through operating-system abstractions, including synchronization and communication services [3]. BORPH extends a Unix-style process model to FPGA resources and uses familiar operating-system mechanisms to connect hardware and software processes [4]. These systems demonstrate that a unified programming view is an established design direction, not a new claim of this work.

SPREAD is particularly relevant because it combines a streaming abstraction with partial reconfiguration and supports switching between hardware and software execution, including channel rerouting and context preservation [5]. The present proposal therefore does not claim to introduce hardware/software migration or transparent channels for the first time. It instead concentrates on making the local message contract, mailbox ownership, and cost decomposition explicit for a small or medium embedded SoPC.

RecoBlock and PR-HMPSoC address reusable reconfigurable blocks and dynamically reconfigurable multiprocessor platforms, respectively [6,7]. More recent work includes the Coyote v2 preprint, which raises the abstraction level for data-center FPGA services, and Reconfigurable Stream Network Architecture, which separates programmable stream data movement from computation in heterogeneous accelerators [8,9]. Their deployment settings and objectives differ from the task-state migration problem considered here; no direct resource or latency comparison is inferred.

Table 1. Relationship to representative systems. The distinctions describe emphasis, not claims that prior systems lack all related capabilities.

System	Established emphasis	Relationship to this proposal
LEAP [2]	Latency-insensitive FPGA channels and platform services.	A precedent for hiding low-level placement details; this paper specifies task migration boundaries and ownership.
ReconOS [3]	Hardware/software threads with operating-system services.	A possible integration environment; this paper isolates the task-facing message and state contract.
BORPH [4]	Unix-style FPGA processes and process communication.	A process-level precedent; the present focus is local mailbox preservation during remapping.
SPREAD [5]	Streaming abstraction, hardware/software switching, context and channel management.	Closest conceptual precedent; motivates explicit comparison of snapshots, routing, and state-transfer semantics.
RecoBlock / PR-HMPSoC [6,7]	Reusable blocks and partially reconfigurable multiprocessor organizations.	Platform-level mechanisms on which a common task interface could be implemented.
Coyote v2 / RSN [8,9]	FPGA service abstraction and programmable stream connectivity.	Contemporary architectural context, not matched experimental baselines.

Table 1. Relationship to representative systems. The distinctions describe emphasis, not claims that prior systems lack all related capabilities.

System	Established emphasis	Relationship to this proposal
LEAP [2]	Latency-insensitive FPGA channels and platform services.	A precedent for hiding low-level placement details; this paper specifies task migration boundaries and ownership.
ReconOS [3]	Hardware/software threads with operating-system services.	A possible integration environment; this paper isolates the task-facing message and state contract.
BORPH [4]	Unix-style FPGA processes and process communication.	A process-level precedent; the present focus is local mailbox preservation during remapping.
SPREAD [5]	Streaming abstraction, hardware/software switching, context and channel management.	Closest conceptual precedent; motivates explicit comparison of snapshots, routing, and state-transfer semantics.
RecoBlock / PR-HMPSoC [6,7]	Reusable blocks and partially reconfigurable multiprocessor organizations.	Platform-level mechanisms on which a common task interface could be implemented.
Coyote v2 / RSN [8,9]	FPGA service abstraction and programmable stream connectivity.	Contemporary architectural context, not matched experimental baselines.

3. Materials and Methods: Architectural and Analytical Model

3.1. Task Graph and Assumptions

Let G = (V, E) denote a directed task graph. A task T_i may have a software implementation, a hardware implementation, or both. The mapping m(T_i, t) associates the task at time t with a processor context, a static hardware block, or a partially reconfigurable region (PRR). An edge (T_i, T_j) identifies a logical communication dependency and remains associated with the same task identifiers when either endpoint changes domain.

Migration is restricted to tasks that expose a defined quiescent point and an explicit portable state representation. Arbitrary machine state, a live processor stack, pointers into private address spaces, and undocumented FPGA registers are not assumed to be transferable. For each migratable task, the developer must specify the state schema, version, byte order, allowable message sizes, and the equivalence required between hardware and software implementations. This restriction is necessary to distinguish application-level migration from general process checkpointing.

The analytical method evaluates deterministic formulas using declared inputs, rather than fitting parameters to observations. The reference datapath is 64 bits wide at 100 MHz. Payload and mailbox sizes use binary units: 1 KiB = 1024 bytes and 1 MiB = 1,048,576 bytes. Bandwidth is expressed in decimal MB/s. Fixed overheads and effective rates are sensitivity inputs, not estimates extracted from a prototype. The supplementary Python script records all inputs and reproduces every numerical table and plot.

3.2. Task-Facing Communication Contract

send(dst, buffer, length); receive(src, buffer, capacity)

(1)

Both operations use logical task identifiers. Int-S is the software adapter and Int-H is the hardware adapter. They need not have identical implementation code or wire encodings, but they must preserve the same application-visible interpretation. The adapter resolves a logical operation into hardware-to-hardware (H-H), hardware-to-software (H-S), software-to-hardware (S-H), or software-to-software (S-S) communication.

The baseline contract treats a message as accepted only after the adapter has taken responsibility for its payload. Acceptance is not equivalent to application consumption. Once accepted, the sender must not modify the buffer unless the adapter has copied the data or explicitly returned ownership. The receiver must not consume an incomplete message. Ordering is required per directed edge, rather than globally across unrelated producers. Backpressure is permitted, but it must be visible through completion or readiness signaling.

The sequence and mapping-generation fields in Table 2 are specification refinements, not implemented or formally verified mechanisms. They make a migration boundary auditable: a route acknowledgement identifies the generation it installed, and a resumed task cannot silently accept data addressed to an earlier binding. Exactly-once processing after a crash would additionally require durable commit and recovery semantics, which are outside this study.

3.3. Managers, Adapters, and Static Infrastructure

A software task, UserApp-S, contains application code, Int-S, and an explicit state buffer. A hardware task, UserApp-H, connects through Int-H to an OCM/BRAM-backed local mailbox and a PRR adapter. The software manager owns the global task-to-location map and performs operating-system-visible memory operations. The hardware manager applies low-level routing and adapter configuration. The static infrastructure includes the routing fabric, boundary buffers, and the bridge between programmable logic (PL) and memory.

Figure 1. Conceptual organization of the proposed interface, redrawn from the source architecture. Solid arrows represent data paths; dashed arrows represent control. The schematic does not imply a synthesized netlist or a measured implementation.

Figure 1. Conceptual organization of the proposed interface, redrawn from the source architecture. Solid arrows represent data paths; dashed arrows represent control. The schematic does not imply a synthesized netlist or a measured implementation.

A basic transaction header retains the original fields IDMD (destination module identifier), LENG (payload length), and IDM (source module identifier). A deployable encoding must also specify field widths, maximum transfer length, error signaling, and whether identifiers name logical tasks or physical bindings at each boundary. The manager translates between the two namespaces. Optional sequence, valid, ownership, and generation fields must be included in the overhead accounting if implemented.

For H-H transfers, routing directs a source mailbox to the destination hardware adapter. H-S and S-H transfers cross the static memory bridge; their consistency therefore depends on both hardware completion and software-visible memory ownership. S-S transfers remain in the platform shared-memory or message-passing subsystem while retaining the logical task namespace. The proposal does not require S-S data to make an unnecessary round trip through the FPGA.

3.4. Interface-Variant Accounting and Resource Model

A task with d neighbors has 2^d possible combinations of peer domains when each neighbor can independently be in software or hardware. This is a count of placement combinations. It becomes a count of interface variants only for a design that specializes the task interface separately for every combination. Existing unified APIs can already avoid that specialization. Under the proposed contract, each supported task domain uses one adapter contract; a task with both implementations may therefore still require an Int-S realization and an Int-H realization.

N_peer = 2^d; N_{contract,domain} = 1.

(2)

Let N_r denote the number of regions, N_b the number of boundary buffers, M_i the usable mailbox capacity in bytes, and C_BRAM the usable byte capacity of a BRAM allocation unit. A symbolic resource budget is:

LUT_static = L₀ + N_r(L_IntH + L_adapter) + N_b L_buffer + L_route,

(3)

FF_static = F₀ + N_r(F_IntH + F_adapter) + N_b F_buffer + F_route,

(4)

$${\mathit{BRAM}}_{\mathit{static}}={\sum }_{i=1}^{N_r}\lceil {\displaystyle \frac{M_i}{C_{\mathit{BRAM}}}}\rceil +{\mathit{BRAM}}_{\mathit{metadata}}$$

(5)

These expressions are accounting identities with implementation-dependent terms, not utilization predictions. Depth/width packing, dual-port requirements, and vendor primitives can change the BRAM allocation. A dense region-to-buffer metadata table has N_r N_b entries under that particular organization; sparse mappings can differ. A flat crossbar also has physical routing and timing costs that are not captured by counting table entries. A hierarchical fabric or network-on-chip should be considered when measured congestion or arbitration cost justifies it, rather than at an asserted universal region-count threshold.

4. Migration, State Preservation, and Cache Ownership

4.1. Migration Sequence

The protocol comprises five externally visible phases: detect a placement change, quiesce the affected endpoint, preserve local data and task state, install the new route or hardware module, and resume. Quiescence stops acceptance of new work at a defined boundary; it does not simply stop the clock while a transaction is partially committed. Already accepted input, partially produced output, per-edge sequence state, and internal application state must be either completed or included in the checkpoint.

For a draining strategy, the manager waits until all accepted transactions have completed and a consistent task checkpoint can be formed. For a checkpointing strategy, the manager records both task state and unconsumed mailbox content, together with the logical edge and sequence position to which that content belongs. Restoring only an output mailbox is insufficient for a stateful task with pending input or intermediate computation. The application must define which strategy is legal at each migration point.

Route rebinding is committed only after the new adapter and task-state representation are ready. When a PRR is reconfigured, its external interfaces must remain isolated until configuration and initialization complete. AMD guidance on Dynamic Function eXchange describes the need to manage the static/reconfigurable boundary during module replacement [10]. The hardware manager must not equate an updated route register with completion of a partial-bitstream load.

Three conditional invariants guide implementation: a state buffer has one mutable owner; accepted messages are either consumed or represented in the preserved state; and a new binding becomes visible only after the old binding is quiescent and the replacement is ready. These are design obligations, not proven properties of an executable implementation. In particular, concurrent migrations sharing buffers require serialization or a consistent multi-endpoint coordination rule.

4.2. Software/Hardware Memory Transitions

Software-to-hardware transfer requires a device-accessible buffer and completion of the platform-specific ownership transition. On a noncoherent path, dirty CPU cache lines must be made visible to the device before it reads the buffer, and device writes must be made visible before the CPU resumes access. In a Linux implementation, these responsibilities should be expressed using the DMA mapping and synchronization interfaces rather than by assuming that a CPU pointer is a valid device address. Coherent allocations do not remove the need for ordering barriers around descriptors and ownership flags [11].

The software manager performs these operating-system operations and supplies the resulting device address and validity information to the hardware manager. The hardware manager then controls routing, ready/valid handshakes, and local mailbox ownership. The division is intentional: a hardware routing controller cannot independently infer which CPU cache lines are dirty or whether a userspace address has a stable DMA mapping.

Hardware-to-software migration additionally requires extraction of all state identified by the portable task schema. This can include accumulator contents, loop position, remaining payload length, and mailbox read/write indices. Their correspondence to the software state must be specified and checked; draining a communication buffer alone does not demonstrate functional equivalence of two task implementations.

4.3. Timing Decomposition and Bounds

Let B_p be the number of bytes actually moved during the selected preservation operation, BW_d its effective transfer rate, and T_o the nonoverlapping fixed costs assigned to that operation, including route update, arbitration, and cache-maintenance components when applicable. The first-order estimate is:

T_preserve,est = B_p / BW_d + T_o.

(6)

Equation (6) is not automatically a worst-case bound. A defensible bound requires an upper bound on preserved bytes, a guaranteed minimum service rate, and independently bounded control, arbitration, and cache costs. With those conditions, B_max/BW_min + T_o,max bounds the modeled preservation component. Unbounded backpressure or scheduling delay invalidates that conclusion. Effective bandwidth must include transfer inefficiency and interference represented by the chosen operating condition.

T_migration = T_quiesce + T_serialize + T_preserve + T_PR + T_restore + T_resume,

(7)

T_PR,est = S_bit / BW_cfg + T_driver.

(8)

Equation (7) names phases separately to prevent mailbox draining from being reported as complete migration downtime. Implementations may overlap some phases, but then a measured critical-path model is required rather than adding overlapping intervals. T_PR is zero for a route-only change with no bitstream replacement. The configuration path and driver can materially affect reconfiguration time; ZyCAP provides an example of studying this management layer on Zynq [12]. No configuration throughput from that platform is assumed in the calculations below.

5. Analytical Results

5.1. Placement Combinations and Transport Budgets

Table 3. Placement-combination count under independent binary peer domains. The third column is per supported task domain, not a count of all implementation artifacts.

Task degree d	Peer-domain combinations 2^d	Adapter contracts per domain
1	2	1
2	4	1
3	8	1
4	16	1
8	256	1

Table 3. Placement-combination count under independent binary peer domains. The third column is per supported task domain, not a count of all implementation artifacts.

Task degree d	Peer-domain combinations 2^d	Adapter contracts per domain
1	2	1
2	4	1
3	8	1
4	16	1
8	256	1

The table illustrates why separating task logic from peer placement can simplify interface maintenance. It does not measure developer effort or prove an exponential reduction relative to an existing unified framework. Manager metadata, adapter verification, and state-schema compatibility remain necessary even when application-level branching is removed.

For a W-bit datapath and clock f, the ideal payload rate is Wf/8. With W = 64 and f = 100 MHz, this rate is 800 MB/s before stalls, headers, or bus inefficiencies. A 1 KiB payload requires 128 data cycles. The following control-cycle budgets are retained as illustrative architectural inputs, not calibrated timing constants.

Table 4. Transport-only illustrative budget for a 1 KiB payload, 64-bit datapath, and 100 MHz clock. CPU cache maintenance, software scheduling, and driver overhead are excluded.

Path	Data cycles	Assumed control cycles	Calculated transport time
H-H	128	12	1.40 microseconds
H-S	128	18	1.46 microseconds
S-H	128	24	1.52 microseconds
S-S	Platform dependent	Platform dependent	Not calculated

Table 4. Transport-only illustrative budget for a 1 KiB payload, 64-bit datapath, and 100 MHz clock. CPU cache maintenance, software scheduling, and driver overhead are excluded.

Path	Data cycles	Assumed control cycles	Calculated transport time
H-H	128	12	1.40 microseconds
H-S	128	18	1.46 microseconds
S-H	128	24	1.52 microseconds
S-S	Platform dependent	Platform dependent	Not calculated

For the three modeled paths, T_transport = (128 + C_control)/100 MHz. Their numerical ordering is a consequence of the selected cycle inputs, not evidence that these relative latencies will occur on a board. In particular, a noncoherent S-H or H-S transfer may incur additional cache and software costs larger than the transport component. S-S requires a processor-side measurement and is not assigned a PL-cycle estimate.

5.2. Mailbox Preservation Sensitivity

Table 5 evaluates Equation (6) for pending mailbox data of 1, 4, 16, and 64 KiB, effective rates of 200, 400, and 800 MB/s, and T_o = 1 microsecond. For this sweep, B_p represents mailbox bytes only; separately serialized internal state and the other terms of Equation (7) are excluded. The 800 MB/s condition is an optimistic reference equal to the ideal datapath payload rate.

Figure 2. Equation-based preservation sensitivity to pending mailbox data and assumed effective drain bandwidth, with T_o = 1 microsecond. The curves are not observations and exclude partial reconfiguration and other migration phases.

Figure 2. Equation-based preservation sensitivity to pending mailbox data and assumed effective drain bandwidth, with T_o = 1 microsecond. The curves are not observations and exclude partial reconfiguration and other migration phases.

At 16 KiB, changing the assumed effective rate from 800 to 200 MB/s increases the preservation estimate from 21.48 to 82.92 microseconds. The difference follows directly from serialization of a fixed byte count. The supplementary sweep also evaluates fixed overheads of 5 and 20 microseconds: at 16 KiB and 800 MB/s, the corresponding estimates are 25.48 and 40.48 microseconds. Reporting the fixed term separately therefore matters when mailboxes are small.

5.3. Configuration-Transfer Sensitivity

A separate sensitivity calculation uses a hypothetical partial bitstream of 1 MiB and effective configuration rates of 50, 100, 200, and 400 MB/s. These rates are illustrative independent inputs, not claimed capabilities of a particular PCAP, ICAP, device, or driver. Setting T_driver = 0 isolates the transfer contribution and does not mean driver overhead vanishes in an implementation.

Table 6. Calculated transfer component for a hypothetical 1 MiB partial bitstream. Driver, initialization, and other migration costs are excluded.

Assumed configuration rate (MB/s)	Calculated bitstream-transfer time (ms)
50	20.97152
100	10.48576
200	5.24288
400	2.62144

Table 6. Calculated transfer component for a hypothetical 1 MiB partial bitstream. Driver, initialization, and other migration costs are excluded.

Assumed configuration rate (MB/s)	Calculated bitstream-transfer time (ms)
50	20.97152
100	10.48576
200	5.24288
400	2.62144

Figure 3. Equation-based partial-bitstream transfer time versus assumed effective bandwidth for a 1 MiB bitstream. The input size and rates are hypothetical and are not measurements.

Figure 3. Equation-based partial-bitstream transfer time versus assumed effective bandwidth for a 1 MiB bitstream. The input size and rates are hypothetical and are not measurements.

For the 400 MB/s configuration scenario and the 16 KiB, 800 MB/s preservation scenario, the sum of these two components is 2.64292 ms. This is not a complete migration estimate: quiescence, serialization, configuration-driver overhead, restoration, and resumption are still missing. The calculation demonstrates the importance of disaggregating components, rather than making a general claim that all migrations take microseconds or that configuration always dominates.

5.4. Reproducibility and Interpretation

The supplementary implementation uses deterministic arithmetic and exports preservation_sweep.csv, configuration_sweep.csv, transport_budgets.csv, variant_counts.csv, and assumptions.json. It includes checks of representative values and generates the plotted curves. Reproduction establishes arithmetic consistency only. It neither validates the architectural safety obligations nor establishes predictive accuracy against a real device.

6. Prospective Implementation and Validation Protocol

A practical first prototype is a three-task pipeline with a migratable middle task and software/hardware versions that implement the same state schema. Frames should include an edge identifier, sequence number, and payload checksum. A reference run without migration establishes expected output. Migration runs should vary the trigger point, pending input and output, mailbox depth, transfer direction, and whether a partial bitstream is loaded. The outcomes must include functional outputs and the fate of every accepted message, not only elapsed time.

Table 7. Proposed validation matrix. No outcomes from these tests are reported in this paper.

Question	Measurement or test	Required evidence
Contract correctness	H-H, H-S, S-H, S-S transfers; migration with in-flight frames.	Sequence/ownership traces; missing, duplicate, reordered, and corrupted frame counts.
State equivalence	Compare uninterrupted and migrated task executions at declared quiescent points.	Application output agreement and a documented state-schema mapping.
Static infrastructure cost	Synthesize and place-and-route candidate region counts such as 2, 4, and 8.	Device/tool versions, LUT/FF/BRAM, timing closure, routing utilization.
Path latency	Instrument PL transfers and processor-side completion separately.	Message-size sweeps, repetitions, contention conditions, distributions and maxima.
Migration downtime	Timestamp each phase of Equation (7).	Quiescence-to-first-valid-output time plus nonoverlapping phase breakdown.
Failure handling	Interrupt checkpoint completion or reject an unsuccessful configuration.	No premature activation; explicit manager failure indication and recovery policy.

Table 7. Proposed validation matrix. No outcomes from these tests are reported in this paper.

Question	Measurement or test	Required evidence
Contract correctness	H-H, H-S, S-H, S-S transfers; migration with in-flight frames.	Sequence/ownership traces; missing, duplicate, reordered, and corrupted frame counts.
State equivalence	Compare uninterrupted and migrated task executions at declared quiescent points.	Application output agreement and a documented state-schema mapping.
Static infrastructure cost	Synthesize and place-and-route candidate region counts such as 2, 4, and 8.	Device/tool versions, LUT/FF/BRAM, timing closure, routing utilization.
Path latency	Instrument PL transfers and processor-side completion separately.	Message-size sweeps, repetitions, contention conditions, distributions and maxima.
Migration downtime	Timestamp each phase of Equation (7).	Quiescence-to-first-valid-output time plus nonoverlapping phase breakdown.
Failure handling	Interrupt checkpoint completion or reject an unsuccessful configuration.	No premature activation; explicit manager failure indication and recovery policy.

Cache tests should include device writes followed by CPU reads, CPU writes followed by device reads, buffer reuse, and transfers that span cache-line boundaries. A correctly configured coherent path and a correctly synchronized noncoherent path should be evaluated separately. Driver versions and memory allocation choices are part of the experimental configuration. Maximum observed latency should not be labeled a worst-case execution-time bound without an accompanying service and scheduling argument.

A comparison with another framework must match the task workload, payload size, state semantics, device, and reconfiguration policy. Otherwise, differences may reflect compiler settings, driver design, or the amount of state preserved rather than interface architecture. Candidate baselines include a fixed-placement implementation and a placement-specific adapter design on the same platform; reproducing a prior framework is a separate engineering task.

7. Discussion and Limitations

An illustrative advanced driver-assistance workload contains rear-camera and forward-camera processing tasks whose priorities change with operating mode. A placement manager could move a suitable kernel between software and FPGA fabric while preserving logical connections. This example motivates route independence but is not a deployed automotive result. Safety deadlines, task equivalence, and the placement policy itself are not established by the communication contract.

Local preservation can reduce the amount of state that must be moved, but it does not guarantee that the rest of the application is unaffected. Backpressure can propagate along dependencies; shared DDR bandwidth can interfere with other tasks; and a task may not reach a quiescent point promptly. The proposed interface leaves scheduling policy outside its scope and exposes the information needed to evaluate these effects rather than claiming to eliminate them.

The principal limitation is the absence of an RTL/software prototype with board-level evidence. Resource equations contain unknown implementation costs, cycle budgets are assumed, and the migration invariants have not been model-checked. The flat interconnect is a starting topology, not a scalability result. A second limitation is portability: a task-state representation and an application-defined quiescent boundary must be engineered for each migratable task. These restrictions are central to interpreting the proposal, not incidental experimental omissions.

8. Conclusions

This work specifies a domain-transparent Int-S/Int-H communication architecture for heterogeneous SoPCs. It preserves logical task identities while assigning routing, local mailbox management, and software-visible memory ownership to cooperating managers. The analysis distinguishes placement-combination counts from actual implementation variants and separates transport budgets, preservation costs, and configuration transfer. Reproducible numerical examples show the dependence of these components on explicitly assumed rates and state sizes. They do not establish implementation performance or fault-tolerant migration. The next step is an instrumented prototype that checks task-state equivalence and message preservation while reporting complete migration downtime, resource use, and contention-sensitive path latency.