Simulator-Based Digital Twin of a Robotics Laboratory

1. Introduction

Simulator-based digital twins streamline the development, deployment, and operation of robotic and industrial systems. These systems are typically specified with formal behavioral models that rely on hierarchy and abstraction to manage complexity. As a result, both industry and students often favor digital-twin workflows to build monitoring and control software and to quickly deliver tangible results.

However, a major drawback is that design details can be obscured, which makes failures harder to diagnose and correct. Engineers -—and increasingly artificial intelligence (AI) assistants– must therefore understand the underlying formal models and the key technologies in the development toolchain. Building this foundation is essential for efficient systems and for addressing failures effectively.

In robotics, core technologies converge within the digital twin: a robot’s data model, often linked to a simulator, and its control software. Robot development thus includes both the physical robot and its digital counterpart.

Time constraints force most robotics courses to emphasize a limited subset of robot design while still enabling hands-on laboratory work. In this paper, we describe how we designed a course that covers the controllers across the full stack from a single computational model, still allowing practical experience and, critically, providing visibility into the internal workings of robotic systems. The latter supports debugging and fault analysis, though it typically increases learning time. To fit a 150-hour course with 50 contact hours, we introduced a simulator-based digital twin of an elementary mobile robot and a simplified model-to-software framework.

The Intelligent Robots course (within an AI undergraduate program) reviews kinematics for wheeled mobile robots and manipulator arms, as well as low-level control, assuming no prior robotics background. The course focuses on behavior modeling with state machines and demonstrates, through code examples, how to achieve real-time simulation, model updates, and synchronization between simulation and physical systems.

The contributions of this work are: (i) a single, simple model that spans all abstraction levels of robotic controllers; (ii) a programming template for synthesizing software from system models; and (iii) a set of models and their software counterparts supporting real-time synchronized simulation of a digital twin that can also act as the controller for the physical robot. Building on [1], we further present a model-based design system that subsumes features of well-known models, including the ability to represent autonomous, intelligent agents, and we provide practical guidelines for synthesizing software manually or with AI assistance.

The paper is organized as follows. Section 2 reviews system-modeling techniques and AI-assisted software generation, as well as related courses and development environments. Section 3 details the critical aspects of robotic controllers in the course setting. We then discuss the course experience and outcomes.

2. State of the Art

Our primary goal was to create a technology-agnostic, model-based design (MBD) framework that minimizes the learning curve while covering the knowledge and skills required to develop control software for industrial machinery and robots. MBD uses formal, machine-readable models that can be simulated, verified, and deployed atop middleware that links application software with visualization, monitoring tools, simulators, and the actual plant or robot.

Contemporary software development increasingly leverages AI for code generation. Yet large language models (LLMs) do not inherently capture model intent and may produce incomplete or incorrect code, increasing the verification burden. Engineering education must therefore provide practical, end-to-end experience, even in introductory courses, which often means carefully scoped lab activities.

2.1. Model-Based Design

Model-based systems engineering (MBSE) has become essential for coping with heterogeneous hardware, distributed software, real-time constraints, and safety-critical requirements. MBSE captures requirements, architecture, behavior, and verification artifacts across the lifecycle, improving consistency, traceability, and multidisciplinary integration [8,9]. MBD uses such models to represent behavior, enable verification, generate code automatically, and accelerate deployment and maintenance.

Models can be tailored to functional and nonfunctional requirements –for example, in manufacturing automation [2] and satellite systems [3]– and translated to executable platforms such as PauWare [4] that support simulation, execution tracing, and runtime verification. Statecharts [5], a common formalism for concurrent hierarchical EFSMs (HCEFSMs), help designers reason about behavior, explore states comprehensively, and scale to complex systems.

Single-state state machines (SSSMs) are sometimes used to encapsulate behavior designed with different methods. An ASML study [6] reported that SSSMs comprised over 25% of 1,500 industrial models and were relatively stable over time.

While code generation depends on both the model and the tools, many common models can be compiled into a model intermediate representation (MIR) to support efficient scheduling, simulation, and code generation in tools like Simulink and Ptolemy-II [7].

In practice, many engineers are comfortable coding ad hoc state machines but are less familiar with designing software from formal models. Projects evolve, so designs must remain adaptable. We address this by leveraging networks of simple EFSMs that emulate statechart features and can be implemented without specialized libraries or environments.

The framework must also account for ROS 2, now a dominant open middleware in industrial robotics, offering real-time–friendly communication, DDS-based security, cross-platform support, and modularity [10,11]. ROS 2 aligns well with formal modeling frameworks such as Timed Rebeca for verifying concurrency, timing, and safety properties and for bridging analysis, simulation, and deployed code [12,13]. Industry-focused ROS 2 frameworks improve modularity and reuse, support rapid development, and provide unified, real-time–capable interfaces [14]. Simulators –most notably Gazebo (tight ROS integration) and Unity-based XR environments– enable simulation-driven development and digital twins that support early validation, model refinement, and faster commissioning [15]. While this ecosystem increases environment complexity, our work focuses on HCEFSMs implemented with concurrent EFSMs and a direct link to raw code. We also relate HCEFSMs to the hierarchical state machines (HSMs) and behavior trees (BTs) commonly used in ROS 2 projects.

2.2. Software Generation

We distinguish programming from software synthesis. Programming translates a model into a particular language and introduces optimizations; software synthesis prioritizes a systematic translation that preserves traceability from the model to the code.

LLMs can generate code [18,19,20], but may propagate errors and security issues from training data [21]. They tend to perform well on simpler problems, struggle with extended reasoning, and often optimize for compilability over robustness, motivating combined correctness-and-security evaluations such as CodeSecEval [22]. Social and technical biases in automatic generation have been documented [23,24,25]. Automatic translation with the help of LLMs has limited accuracy [26], with success rates ranging from 2 to 47% [27], and is not reliable without thorough human review [28,29], complemented by testing and targeted prompts that reduce errors.

Fine-tuning can improve generation quality at lower computational cost [30,31,32,33], but interactive clarification is typically required [34], and performance degrades with overly long or noisy prompts [35]. While top models achieve student-level exam performance in programming [36,37], human experts still excel at structured reasoning and quality control.

Our approach converts formal state-machine diagrams directly into executable code, reducing ambiguity, bias, and translation errors. LLMs can then be used as transducers given paired examples (diagram → code), but they may insert “false optimizations” (e.g., reorganizing code or adding unnecessary initializations) that mask model flaws. We therefore prefer synthesis patterns that maximize model–code traceability and embed self-diagnostics.

2.3. Robotics Education

According to Educations.com, there are 127 undergraduate AI programs worldwide (76 in Europe, 26 in the UK, and 6 in Spain), including ours at the Universitat Autònoma de Barcelona. Many programs include a 6 ECTS (≈150 hours) robotics course with ≈50 contact hours. Our course must introduce robotics fundamentals [38,39] while focusing on intelligent robots [40], covering kinematics and dynamics, controllers, perception (notably vision), execution of predefined plans, SLAM and Kalman filtering, and AI links for perception and decision making—topics that often demand an entire course [41,42] or a dedicated master’s program.

Leading universities (e.g., Oxford, Stanford, UC3M, EPFL) offer compatible overviews, while others (e.g., Berkeley, Imperial, Harvard, NTU, NUS, Caltech, and UPC) provide specialized courses in robotics and AI. Affordable robot platforms enable rich student labs [43]. Digital twins extend this work beyond the lab with safe, risk-free testing. Popular simulators –Player/Stage Gazebo [44], CoppeliaSim [45], and Webots [46]– offer model libraries and support building twins for the lab robots. In our case, we use in-house developed mobile robots and their digital twins that consist of the corresponding robot model and a few high-level data (Figure 1). We also have an arm robot from UR for which we use the built-in model in CoppeliaSim.

Low-level controllers can directly drive actuators, but complex procedures are best programmed atop ROS [47]. In fact, some introductory courses on robotics use ROS [42] and there are educational resources based on ROS [48,49,50]. While we include only a practical ROS example, our programming model mirrors ROS node organization and can generate ROS code. Unlike ROS pub–sub timing, our execution can satisfy stricter time requirements.

Inspired by Ptolemy II [51], our computational model represents behavior as a network of EFSMs communicating via signals in a cycle-by-cycle execution, enabling accurate timing. We simplify Ptolemy II to reduce errors and ease code generation and debugging. Prior work has proposed correct-by-construction behaviors with built-in recovery [52] and fault-handling architectures [53].

Our course content is organized around hands-on modules but emphasizes technology-agnostic controller thinking and robust signal handling -—present, out-of-range, and absent—- so that components embed error detection and recovery.

3. Course Contents

The course covers foundational material common to many offerings: kinematic models of wheeled mobile robots (with emphasis on differential drive) and robotic arms (including inverse kinematics), as well as open-loop and closed-loop control with PID controllers. It also introduces image processing (OpenCV [54]), AI-based detection (YOLO [55]), SLAM, and the Kalman filter.

The differentiating element is the software development process: behavior is modeled as an EFSM network and paired with an ad hoc simulator so students can see how systems work internally and where failures originate. The remainder of this section details the computational model and the real-time execution model used to synchronize the digital twin with reality.

3.1. Behavior Modeling

Because EFSMs underlie more complex models (statecharts, HSMs, and BTs), any of these can be transformed into EFSMs. We thus represent the system as a set of EFSMs executing in parallel. Unlike plain FSMs, EFSMs carry an extended state: a control state plus a data store retained across cycles.

Figure 2 shows a programmable counter with clock input, two inputs (P and b), one output (e), and an internal counter C. Diagram labels define transition conditions and associated actions. Decision nodes (e.g., $C>0$) partition cases to ensure coverage. All activated actions are conceptually concurrent within a cycle. Delayed assignments (e.g., $C^{+}=P-1$) update at the next cycle, while the next control state is determined by the graphical transitions. Designers must avoid conflicting assignments to the same signal (e.g., $e=\mathit{false}$ and $e=\mathit{true}$ simultaneously).

Hierarchy arises when a state contains a subordinate machine (Figure 3, top). In our implementation (Figure 3, bottom), we simplify semantics by running master and slave EFSMs in parallel, sharing variables and I/O and coordinating with active/ready handshake signals. This pattern reproduces statechart behavior while making priorities and continuations explicit and observable.

Arrows coming from composite states that do not originate from stop states of the subordinate machine like $Cx$ can be assigned various meanings, since it is necessary to determine whether the actions of the subordinate machine should take effect and with what priority in case of conflict and also whether, upon returning to the master state, the subordinate machine will remember the current state or start from the initial one again. In the format of the diagrams at the top of Figure 3, this should be specified explicitly, as is done, for example, in statecharts.

To simplify, the hierarchy is implemented so that the slave machines work in parallel with the masters, sharing variables and input and output. The hierarchy is established through a handshake protocol with activation and readiness signals. Thus, when the master state machine reaches the SUB state, the active signal is set to true so that the slave machine exits the IDLE state in the next cycle.

When the slave machine returns to IDLE, the ready signal returns to true so that the master can continue. This reproduces the same behavior as the arrival at END in the slave machine depicted above and the exit arc from the octagon in the composite state of the master machine.

The semantics associated with exit arcs that do not wait for the subordinate EFSM to return to an inactive state can be easily simulated in the composite state part, with an extra decision node, as can be seen in the lower right part of Figure 3. On the subordinate machine side, the designer must consider all situations and decide, for each state $S_J$, whether to suspend the corresponding activity and add a decision node active and, in that case, whether to restart the activity in that state or return to IDLE. In this way, there are no errors in interpreting which actions are taken. It should be noted that it can be monitored when the subordinate machine continues with the activity and the master has exited SUB because the ready signal of the subordinate will remain at false.

Thus, concurrent EFSMs can represent hierarchical machine behaviors such as statecharts and HSMs and BTs, which are more common in the field of robotics. In fact, in the case of BTs, it is even simpler, since the various nodes represent communication between machines working in parallel.

For intelligent behaviors, we extend EFSMs with a state stack (EFS2M) inspired by pushdown automata [56]. The EFS2M model adds a state stack and push, pop, and top operations and the corresponding graphical representation.

The EFS2M in Figure 4 generates a plan $P1$ in case plan1() returns true, which consists of following a sequence of states, from $P{1}_1$ to $P{1}_{N1}$ and GOAL. The plan appears in reverse order since the bubbles with state names that appear after the tip of an arrow are states that are pushed and, therefore, will be popped in the reverse order to which they were put on the stack.

In state $P{1}_1$ there are three types of output arcs. The one that is born directly from the state bubble and that is activated with next1_1_2() is a regular arc that goes to the state at the top of the stack. That is, the hexagon at the end of the arrow indicates that the next state will be the one that is popped from the stack. The arc labeled with skip1_2() is born after a bubble that represents a pop operation on the state stack. If you want to indicate that it is completely emptied, you do not need to put as many bubbles as there are states on the stack, but it is enough to put a hexagon there, as shown in the arc abort1_1().

This model naturally represents the planning style of belief-desire-intention(BDI) agents [57], where intentions correspond to stacked plans and desires to the initial pushed states.

As mentioned, the computational model used is an EFSM network with implicit hierarchy, that is, the model does not include any explicit indication of subsumption of one component to another. The description of such a model includes that of the architecture of its system and that of the EFSMs it contains. The architecture diagram also indicates how the different components relate to each other and can be hierarchical. In this sense, this computational model is comparable to behavioral trees.

At the architectural level (Figure 5), we adopt a conventional three-tier stack for a mobile robot. L2 (deliberation) can host a BDI-like agent but, in our course, serves as a user-facing interface that selects missions (SWEEP, FOLLOW, NAVIGATE). L1 (execution) decomposes missions into L0 (reactive) primitives.

Each architectural element expands to one or more EFSM, i.e. multiple EFSMs may coexist per layer. Outputs are driven from state variables (Moore semantics) to prevent circular dependencies and order-sensitive updates. For example, Figure 6 shows the LIDAR sweep state machine part of the main state machine in L1.

3.2. EFS2M Software Generation

Modern control/embedded environments provide modeling, simulation, verification, and code generation—often bound to a specific middleware. A key advantage of EFSM/EFS2M networks is that their behavior can be coded directly in any language without special libraries.

3.2.1. EFS2M Programming Template

There are several ways to translate EFS2M diagrams into programs. They can be classified into those based on state machine interpreters and those that are hard coded. In the first case, the transition and output tables of the EFS2Ms are interpreted by a program that simulates them. In the second case, the simulator program calls specific functions for each EFS2M.

In our case, each state machine is an object of a class with methods to initialize (init) it and simulate its behavior. For the latter, it is divided into four phases: reading inputs (read_inputs), calculating the next extended state (plan), updating it (update), and writing outputs (write_outputs).

The programming of the input/output functions must be done specifically because they depend on the input and output devices used. For example, for the low-level code of the virtual robot, the input and output are via CoppeliaSim.

However, the functions of the methods init, plan and update depend only on the state machine, and therefore the code can be generated following a pattern of correspondence between the elements of the diagram and instructions for the corresponding method.

The patterns for these functions assume that all variables in the state machines are tuples of two values: the current value and the value in the next state. (This implementation is not the most efficient in terms of execution cost, but it is one of the most efficient in terms of engineering cost.)

The state stack is another component of the main state variable tuple that is only manipulated in the plan function if the corresponding diagram contains some element that indicates it, such as push and pop bubbles or empty or pop hexagons (to change to the state at the top of the stack).

These programming patterns are almost templates in which programmers only have to fill in the gaps. However, it is common to incorporate false optimizations [58] that, at best, have no serious effects such as masking coding errors or in the model.

Furthermore, there is no guarantee that premature optimizations are necessary, or that some micro-optimizations such as changes in the format of some data may have an effect on the final executable code, or that code that takes advantage of specific features of a programming language will be more efficient in terms of memory or time than code based on more general instructions.

Basically, it is not a question of programming state machines but of synthesizing their code and, therefore, respecting as much as possible the link between each element of the state machine diagram and the code that corresponds to it.

In this way, it is also possible to incorporate mechanisms that check, for example, if no transition is activated or if more than one is activated in a given cycle, or if an unexpected state is reached.

In this sense, the course insists on the fact that any error in the model must have associated code that reveals it. For example, the monitor function must allow the observation of variables, even if they do not have a defined value.

3.2.2. Software Synthesis

Given that EFS2M programming follows well-defined patterns, it is feasible to automate the process, which also avoids the problem of false optimizations.

Automatic code synthesis can be done algorithmically or with the support of AI. In fact, an EFS2M software generator was already available that read the XML of the models and generated the corresponding Lua code. However, the graphical representations of the models must be syntactically correct for the synthesis program to generate code and, in addition, variations in the input format and output language can imply significant modifications with a large associated cost.

The rise of programming assistants based on LLMs takes advantage of the fact that there is a good database of both algorithmic solutions to problems and programming of these algorithms. As has been seen, this does not mean that the generated code must be reviewed extensively.

However, in the particular case of behavioral models, it is not necessary for the LLM to be trained to correlate problems and algorithmic solutions but rather to correlate models with codes that follow a specific pattern.

In this sense, a series of examples have been created that pair EFS2M and the corresponding programs so that the AI assistant can generate an appropriate output for the target model that has a very high correlation with the given examples.

In experiments with several AI chats (ChatGPT, Copilot and DeepSeek) it has been observed that they all work well for simple cases where the input is a syntactically correct diagram. In fact, in these cases it is irrelevant whether the input is an image of the state machine or an XML or the output programming language.

In the case of images, synthesis works better because it does not depend on attributes of the objects in the drawing and, for example, arrow connections are established by proximity while, in the XML format, it depends on whether the two objects are linked or not.

It has also been observed that they tend to introduce false optimizations and corrections such as initializations not present in the models, which can mask behavioral errors.

In short, software synthesis programs generate code without false optimizations and, therefore, with reliable self-diagnosis, but they require inputs with clear graphical syntax and must adapt to each input and output format. AI chats, on the other hand, can work with any input and output, as long as the instructions include examples in the formats used, but they can reproduce the same problems that could be encountered with human programmers.

3.3. Real-Time Simulation

The execution model of state machines can be represented by a state machine (Figure 7) that cyclically repeats the phases of reading input data (SENSE), calculating the next extended state (PLAN), updating it (UPDATE) therefore moving to the next cycle, and writing primary outputs (ACT). This means that all machines in the system are always in the same phase.

This execution model links, to each state, the call to a specific function of all the state machines of the system and is easy to program in any programming language, which allows the implementation of the system simulation in any development environment.

In this EFSM, the variable C is a cycle counter, that is, it indicates how many transitions the state machines have made until the current cycle, the variable $sT$ is the simulated time, which starts at 0 and the inputs T and $dT$ provide the real time of the system where it is executed and the time step in each cycle. For simplicity, this time step is fixed and the condition for it to be executed in real time is that the execution time of each cycle is always less than or equal to $dT$.

To calculate the time elapsed in a cycle, there is the variable B (for “begin time”) that stores the reference of time at which each cycle begins. The state machine will remain in IDLE until $T-B\ge dT$. In this case, it has been chosen to increase B by $dT$ instead of assigning it to the value of T to allow the excess execution time of previous cycles to be compensated for with subsequent cycles in which the system runs faster. However, it must be considered that there are systems that do not tolerate this soft real-time behavior.

In systems that work with very short execution times compared to $dT$, more than one cycle can be advanced per time step. In this case, cycles will be advanced until some calculation of the next state of some state machine of the system requires an external input. This information is stored in the variable X upon reaching the UPDATE state from PLAN, which is where the function step is called and returns this condition.

The variable I indicates the cycle number within the current period, so that signals that are events cease to be present when $I>0$. In this sense, this must be taken into account when programming the read_inputs functions, since they are the ones that must distinguish between inputs that are events and others that are continuously present.

3.4. Simulation and Reality Synchronization

The simulated robot’s controller stack can be used for the real robot if it has part of the stack embedded and can communicate with the upper part. In our case (Figure 8, the stack is divided between L0 and L1 and, in addition, communication is maintained with both the virtual robot and the real robot so that the system can have both robots in operation.

For simplicity, when the real robot is connected, the responses sent to L1 are those of the real robot. This can generate discrepancies between the simulation and reality, since the simulated environment does not have to coincide with the real one, and there are no mechanisms implemented to make this update.

However, the fact that development can be completed in the simulated environment and that, in the laboratory, the connection with the real robot is transparent to the model, makes the experience more positive.

3.5. Model Update

One of the most common uses of AI is to analyze data to find patterns and features that are difficult to extract with algorithmic methods. This is also true for identifying parameters in controllers’ models. The course does not go so far and considers how a data logger for the robot should be made so that the data in the robot behavior model can be updated, and to detect system errors.

Various data loggers are considered, from those that store everything to smarter ones that only record events of interest, whether errors or pre-processed data to update the model parameters. The smart data logger presented in Figure 9 stores only data related to the robot’s motion operations to be able to determine how well the real-time simulation adapts to the real robot and alarms related to out-of-context messages and timeout situations.

In the log file, records are stored labeled with the real time (T), the rotation and displacement that it had to make and the time at which the first response (lap1) of one of the robots and the second (lap2) of the other was registered. To know which one it is, you need to see if the state is AHEAD (the first response comes from the virtual robot), BEHIND (the first response comes from the real robot) or SYNC, in which they arrived at the same time. In case the responses are not coherent or if any of them arrive after a certain time, an error is generated.

4. Conclusions

Developing software for industrial monitoring, control, and robotics requires substantial knowledge and skill. Simulator-based digital twins help practitioners focus on behavior, but can hide details that matter when systems fail. We presented a development environment that couples a digital twin with a transparent computational model: any system behavior can be described as an EFS2M network.

Compared with statecharts, HSMs, and BTs, the proposed modeling style is simpler to learn, robust by construction, and easy to implement without specialized middleware. To be widely useful, however, it should align with the tools most engineers use, notably ROS 2. Our initial motivation was pedagogical: cover a broad, practice-oriented spectrum so students can see and reason about the full path from model to executable controller, both in simulation and on the real robot.

Over three years of teaching, students reported positive hands-on experiences and demonstrated the ability to adapt to projects using different modeling styles and toolchains, while requesting greater design freedom. From an educational standpoint, future work should include parallel examples implemented in other environments to ease transfer. From a research perspective, we will continue to embed self-diagnosis and error management in the generated code to detect both model and synthesis issues early.