Case Studies of Formal Verification of Task Management Modules in Real-Time OS Kernel Using Bounded Model Checking

1. Introduction

Currently, the scale of embedded systems is increasing significantly, and real-time operating systems (RTOS) are indispensable for embedded systems. RTOS (Real-Time Operating System) is an operating system with strict time constraints, and depending on the system it is installed on, violations of these time constraints can lead to serious accidents. Furthermore, RTOSs implement a task-based system that divides each system process into tasks and schedules them, thereby improving CPU execution efficiency. Therefore, the correctness of task scheduling is crucial for meeting the system’s time constraints.

In mutual exclusion control between tasks using semaphores, the task priority inversion phenomenon [1] may occur. When the priority Inversion phenomenon occurs, tasks with lower priority are processed before those with higher priority, making it easy to miss time constraints. Currently, the validity of the kernel is guaranteed through thorough testing and simulation. Testing and simulation have a limited number of system inputs and cannot simulate all possible behaviors the system may exhibit. Therefore, this paper presents a new verification case for the task management module of an RTOS using a pre-existing bounded type model checker called CBMC [3], which fully supports C/C++ and is based on a SAT solver, and a model checker called ESBMC [10], which is based on an SMT solver. Through experimentation, we were able to formally verify the correctness of task scheduling processes and the occurrence of priority inversion phenomena. This ensures the correctness of the system’s scheduling and is effective for detecting realistic time constraint violations. The RTOS used the ASP3 kernel compliant with the ITRON specification [4], which is publicly available as TOPPERS (Toyohashi OPen Platform for Embedded Software) [2].

The structure of this paper is as follows: Section 2 explains the fundamental concepts of this research, Section 3 presents specific verification cases, Section 4 describes the experimental methods, and Section 5 presents the experimental results. Finally, Section 6 provides a summary and future directions.

2. Preliminaries

2.1. Real-Time Operating Systems

A real-time operating system (RTOS) is an operating system with strict time constraints, distinct from general-purpose operating systems (OS), and is indispensable for today’s large-scale embedded systems. The primary functions of an operating system include task management, memory management, file management, management of peripheral devices such as keyboards and mice, and providing APIs to applications. Furthermore, the real-time OS used in this study is the TOPPERS/ASP3 kernel, which conforms to the $\mu$ITRON 4.0 specification and is publicly available at TOPPERS [2].

The following explains the implementation of the task management module and semaphores in the TOPPERS/ASP3 kernel.

2.1.1. Task Management Module

An RTOS has the capability to divide each system function into tasks. By dividing tasks into smaller units and having the OS schedule their execution, CPU utilization efficiency can be improved. Additionally, users can develop processes within a task without being aware of other tasks. An RTOS primarily manages tasks in four states: running, ready, waiting, and suspended. Additionally, the RTOS scheduler employs rate monotonic scheduling (RMS) [6]. RMS is an optimal scheduling algorithm that uses preemptive fixed priority. High-priority tasks generally have strict time constraints. Therefore, it is clear that ensuring the correctness of scheduling processes—one of the main subjects of this paper—is critically important from the perspective of system safety.

Here, the task initialization block for the TOPPERS/ASP3 kernel is shown in Table 1, where tskatr stores whether a task is initialized in the active state or the suspended state; it is set to $TA\_ACT$ when initialized in the active state and $TA\_NULL$ when initialized in the suspended state, and exinf sets the arguments passed to the task.

Next, the kernel’s task management functions are shown in Table 2. Note that Table 2 lists only representative features; in reality, it possesses various other functions as well.

Figure 1 illustrates the state transitions of the task for each call in Table 2.

2.1.2. Semaphore

A semaphore is an object used for mutual exclusion, synchronization, and communication between tasks, consisting of a semaphore variable and a wait queue. Semaphore functions include initialization ($initialize\_sem$), resource acquisition ($wait\_sem$), and resource release ($signal\_sem$). When the semaphore variable is 1 or greater, $wai\_sem$ assumes resources are available and continues processing the task that called $wai\_sem$. When the semaphore variable is less than 1, resources are not available, so insert the task that called $wait\_sem$ into the semaphore ready queue and call the scheduler. $sig\_sem$ makes the task at the head of the semaphore ready queue executable and schedules it when the semaphore ready queue is not empty. When the semaphore is empty, increment the semaphore variable and resume execution of the task that called $sig\_sem$. The semaphore initialization block in the TOPPERS/ASP3 kernel is shown in Table 3, and the semaphore management block is shown in Table 4. Here, setting the semaphore attribute to $TA\_TPRI$ causes insertions into the ready queue to occur in priority order. In $initialize\_sem$, the $wait\_queue$ is initialized, and $semaphore\_control\_block{`}^{\prime }s$$semcnt$ is assigned the value of $semaphore\_initialization\_block{`}^{\prime }s$$isemcnt$.

2.2. Priority Inversion

Many RTOSs use rate monotonic scheduling [6]. In other words, when resource contention occurs between tasks, the scheduler grants access to the task with the highest priority among the contending tasks. Priority inversion [1] refers to the phenomenon where a high-priority task is forced to wait for a low-priority task to complete. High-priority tasks generally have strict time constraints, and priority inversion can lead to missed deadlines, making this phenomenon a critical factor that compromises system safety. There are two types of priority inversion phenomena as follows:

Restricted inversion

• A low-priority task locks a resource using a semaphore or similar mechanism and enters a critical section. Subsequently, execution shifts to the high-priority task, which enters a waiting state while attempting to access the locked resource. Only after the low-priority task releases the resource does the high-priority task begin execution. This is called restricted inversion because priority inversion occurs only when low-priority tasks are executing within critical sections.

Unbounded inversionl

• Just as with restricted inversion, if a high-priority task is kept waiting while a low-priority task continues to switch execution to other tasks during its critical section, the high-priority task will be kept waiting unboundedly for resources to be released.

When estimating task execution time in the system, it is necessary to take into account objects that may interfere with task execution. Objects that prevent task execution should only be tasks with higher priority than themselves, provided that priority inversion does not occur. However, this is not the case in systems where the priority inversion phenomenon occurs. In restricted inversion, only the task holding the resource prevents the task from executing, but in unbounded inversion, all tasks must be considered. This can pose a significant problem when estimating the system’s worst-case execution time, and above all, it increases the likelihood of time constraint errors.

As a solution to the priority inversion problem, while restricted inversions are unavoidable, it is possible to mitigate unbounded inversions to the level of restricted inversions. This method is called the priority inheritance protocol. This is a method for increasing the priority of a task to the maximum value of the priority of the original task and the task delayed by it. This prevents unlimited inversion, allowing only limited inversion. Mutexes are objects for mutual exclusion control that employ a priority inheritance protocol. Unlike semaphores, mutexes cannot manage multiple resources during mutual exclusion. Therefore, when the priority inversion phenomenon is detected using the proposed method in this paper, system designers must choose whether to use mutexes to avoid unbounded inversions—though this only manages a single resource—or to continue using semaphores even if unbounded inversions occur.

2.3. Software Model Checking

Software model checking [7] is a technique that describes a program as a model and verification properties as temporal logic formulas. By inputting the model and verification properties into a model checker, the model checker mechanically and exhaustively examines whether the verification properties hold or fail in all possible states and paths the model can take. Model checking is superior to simulations that consider only specific inputs because it comprehensively examines all states.

Next, we describe the SAT/SMT solvers used for logical formula satisfiability determination in software model checking.

2.3.1. SAT/SMT Solver

A SAT (boolean SATisfiability problem) solver is a machine that determines the satisfiability of a given propositional logic statement [9]. SAT solvers can only handle propositional logic. When using SAT solvers for program verification, the program must be symbolically represented in propositional logic. Depending on the program’s size, this can cause the problem size to explode, making verification impossible. Furthermore, when encoding programs as logical expressions, it is easier to handle them when encoded as predicate logic such as $x<0$. Therefore, an SMT (Satisfiable Modulo Theories) solver is a machine that determines the satisfiability of predicate logic, and it is composed of an SAT solver and a theory solver consisting of integer arithmetic theory, array theory, pointer theory, and so on. Even for problems where the problem size explodes in SAT solvers, SMT solvers can keep the problem size manageable.

For example, when verifying with an SMT solver, suppose we encode the program into predicate logic as shown in the following equation (1).

$$x\le 0\wedge y\le 0\wedge \neg (x\ge 1)\wedge x>y$$

(1)

First, the SMT solver replaces $x\le 0$, $y\le 0$, $x\ge 1$, and $x>y$ with propositional formulas P, Q, R, and S, respectively, yielding the following formula (2).

$$P\wedge Q\wedge \neg R\wedge S$$

(2)

Since this is propositional logic, it can be solved using an SAT solver. Therefore, if solving (2) with an SAT solver yields a satisfying solution where $P=True,Q=True,R=False,S=True$, then we map back from propositional logic to predicate logic as follows: $P=True$ implies $x\le 0$, $Q=True$ implies $y\le 0$, $R=False$ implies the negation of $x\ge 1$, i.e., $x<1$, and $S=True$ implies $x>y$. Then, by inputting this predicate into the solver, the SMT solver outputs that the original predicate reasoning is satisfiable at $x=-1,y=0$.

2.3.2. Bounded Model Checking

Model checking may fail to verify large-scale programs due to the state explosion, as it exhaustively searches all possible states. Therefore, bounded model checking [8] is a model checking technique that verifies by considering only a certain number of transitions, thereby suppressing the state explosion. Below, we describe specific verification methods for bounded model checking [10]. A state transition system $M=(S,T,S_0)$ is an abstract machine consisting of a set of states S, where $S_0\subseteq S$ is the set of initial states and $T\subseteq S\times S$ is the transition relation. A state $s\in S$ consists of the program counter $pc$ and the values of all program variables. Here, we assume a transition system M, a constraint $\varphi$, and a fixed length k are given. Bounded model checking expands the transition system k times and transforms $\varphi$ into a verification property $\psi$ that can be satisfied only if $\varphi$ has a dual of length at most k. Furthermore, $\psi$ is a first-order logic formula without quantifiers, and its satisfiability is checked by SAT/SMT solvers. The specific logical formula is given by (3).

$${\psi }_k=I\left(s_0\right)\wedge {\displaystyle \underset{i=0}{\overset{k}{\bigvee }}}{\displaystyle \underset{j=0}{\overset{i-1}{\bigwedge }}}\gamma (s_j,s_{j+1})\wedge \neg \varphi \left(s_i\right)$$

(3)

Here, $\varphi$ is the verification constraint, and I is the set of initial states of M. $\gamma (s_j,s_{j+1})$ is the transition relation from step j to step $j+1$ in M. $I\left(s_0\right)\wedge {\bigwedge }_{j=0}^{i-1}\gamma (s_j,s_{j+1})$ represents an execution of length i of M, and satisfies (3) only if there exists an reachable state where $\varphi$ is violated at time step i for some $i\le k$. In other words, when ${\psi }_k$ is satisfied, the system fails to satisfy the verification constraint $\varphi$, and when ${\psi }_k$ is unsatisfiable, the system satisfies the verification constraint.

2.4. C Bounded Model Checker

C Bounded Model Checker (CBMC) [3] is a bounded model checker for C/C++ programs. CBMC converts the program into a bitwise logical expression by expanding the verification constraints specified by the user in assert statements to a user-specified depth k times, then inputs this to an SAT solver for verification.

The CBMC architecture is shown in the following Figure 2[11].

1.

First, I’ll explain the front end. CBMC configures CBMC according to user-specified parameters. In the C Parser section of Figure 2, a C preprocessor (such as gcc) is used to construct a parse tree from the program. This constructed syntax tree is then used for type checking. If a contradiction is detected at this point, verification is terminated. CBMC employs GOTO programs as an intermediate representation. The GOTO program replaces all if statements, switch statements, loops, and jumps with goto statements. Then, a GOTO program is generated from the syntax tree, and at this stage, constraints are added to perform static analysis and detect invalid pointer operations, memory leaks, and other issues.

2.

Next, we explain the middle-end. CBMC converts GOTO statements into static single-assignment (SSA) form while unwinding loops by the number of times specified by the user.

3.

Finally, an explanation of the backend. CBMC converts the SSA-formatted GOTO program generated by the middle-end into a conjunctive normal form (CNF expression). It then inputs the CNF expression into an SAT/SMT solver. If satisfied, it provides the precedent causing the violation.

Additionally, CBMC has the capability to perform memory leak checks and null pointer access checks.

2.4.1. Examples of CBMC Verification

CBMC has several valid verification examples.

1.

For example, there is an effective verification example in real-time systems[12]. This guarantees the timing constraints of periodic embedded systems, an important subclass of embedded real-time systems. Specifically, given a periodic real-time system C and a constraint time W, we construct a sequential program S that approximates all executions of C up to time W, while considering the bound on the number of preemptions implied by priority and rate monotonic scheduling. And by verifying S using CBMC, we guarantee the timing constraints.

2.

Additionally, there are verification examples in systems with interrupt mechanisms [13]. This converts a parallel interrupt-driven program into a sequential program and verifies data races—a critical type of error in interrupt programs—using CBMC.

Thus, CBMC has demonstrated its effectiveness in embedded system verification. However, despite being the core of embedded systems, the task management module of the RTOS kernel has not been verified using CBMC. Furthermore, while the priority inversion phenomenon is common, there are no verified examples.

Therefore, the proposals and verification cases in this paper are novel and can be considered useful.

2.5. Efficient SMT-Based Context-Bounded Model Checker

Efficient SMT-based Context-Bounded Model Checker (ESBMC) [10] is a bounded-type model checker for C/C++ programs. The difference from CBMC is that it uses an SMT solver as the satisfiability checker. The following Figure 3 depicts the ESBMC architecture. ESBMC, like CBMC, possesses the capability to perform memory leak checks and null pointer access checks. Furthermore, in this paper, we added the verification constraints proposed in the next section to the sample program and verified them using both CBMC and ESBMC, conducting comparative experiments between SAT solvers and SMT solvers.

3. Case Studies

3.1. Case Studies on Task Scheduling

In this paper, we propose a method for guaranteeing the correctness of scheduling by implementing a sample program that runs on an RTOS, adding the proposed verification properties, and then verifying it using CBMC/ESBMC. The following section describes the contents of the sample program and the proposed verification constraints.

In order for users to utilize the RTOS kernel’s features within an application, they must statically initialize the kernel features they wish to use. Since this verification focuses on task scheduling, we initialized the task management module within the kernel. As an example in this study, we used three tasks. The specific initialization is shown in Table 5.

Here, TA_ACT is initialized as the active state, and TA_NULL is initialized as the idle state. Additionally, since the arguments for the task are not used in this verification, they were set to 0. In addition, in an RTOS, the priority range is capped at 16, with 1 being the highest priority and 16 being the lowest. In other words, in this verification, TASK 1 has the lowest priority and TASK 3 has the highest priority. The stack size was set to 4096 for all tasks.

Next, we will explain the added constraint. First, we define boolean variables $tsk1$, $tsk2$, and $tsk3$ corresponding to the number of tasks, and initialize them to $false$. Then, at the beginning of each task, we assign $true$ to the respective boolean variables. We then use these variables to define the following equations (4), (), and (). 　

$$\begin{array}{c}\hfill l_1:=tsk1\wedge \neg tsk2\wedge \neg tsk3\end{array}$$

(4)

$$\begin{array}{c}\hfill l_2:=tsk1\wedge tsk2\wedge \neg tsk3\end{array}$$

(5)

$$\begin{array}{c}\hfill l_3:=tsk1\wedge tsk2\wedge tsk3\end{array}$$

(6)

Here, (4) is a constraint indicating that only TASK 1 was executed, () is a constraint indicating that both TASK 1 and TASK 2 were executed, and () is a constraint indicating that all tasks were executed. Here, Figure 5 shows the task details of the sample program and a portion of the source code for the main function, while Figure 4 is a diagram abstracting the execution flow of the sample program.

Figure 4. Execution Flow of Scheduling Verification.

Figure 4. Execution Flow of Scheduling Verification.

Figure 5. Part of the source code for scheduling verification.

Figure 5. Part of the source code for scheduling verification.

The following explains the purpose of scheduling and additional constraints.

1

Since TASK 1 is the only one in an active state at initialization, TASK 1 is executed when the program starts. Therefore, at the start of TASK1, we add $l_1$ using an assert statement.

2

When TASK2 becomes ready for execution via $act\_tsk$(TASK2), execution shifts to TASK2 because it has a higher priority than TASK1. Therefore, we add $l_2$ at the start of TASK2.

3

When TASK3 becomes ready for execution via $act\_tsk$(TASK3), execution transfers to TASK3 because it has a higher priority than TASK2. Since all three tasks have been executed at this point, we add $l_3$ to all remaining points.

Note that, as noted in Section 2.1.1, $act\_tsk\left(\right)$ is a function in the TOPPERS/ASP3 kernel and serves as the task launch function. Figure 5 is abstracted code intended to verify the correctness of the scheduling, so it does not contain any task details. However, this paper guarantees the correctness of the scheduling process in the RTOS kernel. While more complex task transitions are possible in actual systems, all systems call the RTOS kernel during scheduling. Therefore, to ensure the correctness of the actual system’s scheduling, it is sufficient to verify the correctness of the RTOS kernel’s scheduling process using the sample program in this paper.

3.2. Case Studies on Priority Inversion

In this paper, we implement sample programs that demonstrate both restricted inversion and unbounded inversion. We then demonstrate that the proposed method can detect priority inversion by adding the proposed verification constraints and verifying them using CBMC/ESBMC.

Next, we will explain how to implement the sample program. Task initialization is similar to scheduling verification and is shown in Table 5. Furthermore, the initialization of the semaphore is as shown in Table 6.

Since the SEM1 attribute is set to $TA\_TPRI$, tasks are inserted into the semaphore ready queue in order of priority. Since the initial value of the resource is 1, at the start of the program, the resource managed by SEM1 is in a state that allows access by one task. Furthermore, since the maximum number of resources is 1, access to resources managed by SEM1 is restricted to a single task. Furthermore, using the objects from these initialized RTOS kernels, we implemented a sample program for restricted inversion as shown in Figure 6 and a sample program for unbounded inversion as shown in Figure 7.

Next, we will explain the added constraint. First, we define two boolean variables, $task\_first$ and $task\_last$, for each task and initialize them to $false$. $task\_first$ is set to $true$ when the task begins, and $task\_last$ is set to $true$ when the task ends. Using these variables, we define the following expressions (7), (8), and (9).　

$$\begin{array}{c}\hfill l_4:=(task2\_first\wedge \neg task2\_last)\vee \\ \hfill (task3\_first\wedge \neg task3\_last)\end{array}$$

(7)

$$l_5:=(task3\_first\wedge \neg task3\_last)$$

(8)

$$l_6:=(task2\_first\wedge \neg task2\_last)$$

(9)

In this context, a priority inversion occurs when a task with a higher priority is started and executed, but is not completed; instead, execution switches to the lower-priority task midway through, and the higher-priority task is eventually completed. Therefore, we add constraints such as (7), (8), and (9) to the end of all tasks that have tasks with higher priority than themselves. In this verification, since TASK1 has the lowest priority among all tasks, equation (7) is inserted upon completion of TASK1. Since TASK2 has a lower priority than TASK3, insert equation (8) when TASK2 is completed. Additionally, to verify a restricted inversion, we insert the constraints described above immediately before releasing the resource.

Here, Figure 8 abstracts the execution flow of a sample program in which restricted inversion occurs, while Figure 9 abstracts the execution flow of a sample program in which unbounded inversion occurs; both figures indicate the insertion points of the constraint expressions.

The following section explains the execution scheduling of unbounded inversion.

1

Since TASK1 is the only task in an active state at initialization, TASK1 is executed, and TASK1 calls $wai\_sem\left(SEM1\right)$ to lock the resource.

2

When TASK 3 is launched, execution shifts to TASK 3 because its priority is higher than that of TASK 1. Then, when TASK3 attempts to execute $wai\_sem\left(SEM1\right)$, since the resource is locked by TASK1, TASK3 enters a waiting state and execution transfers to TASK1.

3

TASK2 is launched, execution moves to TASK2, and once TASK2 has finished processing, execution returns to TASK1.

4

TASK3 will not execute until TASK1 releases its resources via $sig\_sem\left(SEM1\right)$. (An unbounded inversion is occurring.)

The constraint formula is added at the end of each execution, as shown in Figure 8 and Figure 9. Since the key point of the verification is that the verification fails when a priority inversion occurs, we added the negations of equations (7), (8), and (9) to the program using assert statements.

4. Experimental Methods and Environment

For the experimental methodology, the task scheduling verification described in Section 3 was conducted by increasing the number of tasks to 2, 3, 4, and 5. We also performed memory leak checks and NULL pointer access checks on each of the sample programs described in Section 3. The details of the experimental setup are shown in Table 7.

5. Experimental Results

The verification results are shown in Table 8. The items in Table 8 are as follows:

1.

As "verification item", Verification1 refers to the verification of task scheduling described in Section 3.1, Verification2 refers to the verification of restricted inversion described in Section 3.2, and Verification3 refers to the verification of unbounded inversion.

2.

“Number of Tasks” refers to the number of tasks, "Unwind" refers to the number of transitions k in bounded model checking, and “Time” refers to the verification time.

3.

"Solver" refers to the solver used, which is either an SAT solver or an SMT solver.

4.

In the “Result” , SUC indicates a successful verification; PF (Pointer Failure) indicates a failure due to a NULL pointer access; IF (Inversion Failure) indicates a failure due to a priority inversion; MO (Memory Out) indicates a verification failure due to insufficient memory; and TO indicates a verification failure due to a timeout.

As shown in Table 8, the results of each verification were as follows.

• Verification1
  In verification using the SAT solver (CBMC), the system succeeded in all tasks even when the number of tasks was increased to 2, 3, 4, and 5. This demonstrates that the proposed method described in Section 3.1 can guarantee the correctness of task scheduling. In addition, it is evident that verification time increases as the number of tasks increases. However, in verification using the SMT solver (ESBMC), the result is “TO” for four or more tasks. This is because SMT solvers are more complex than SAT solvers, which is why they take too long to run in this research environment. Furthermore, when examining the results for Task 3, a significant difference in execution time was observed: while the SAT solver took approximately 8 seconds, the SMT solver took approximately 1,214 seconds.
• Verification2
  In the SAT solver, this results in an “IF” condition, and Section 3.2 demonstrates that the method proposed in this study is effective for detecting restricted inversions. However, when we attempted to verify the model using the SMT solver, an internal error occurred in ESBMC, preventing the verification from completing.
• Verification3
  We were unable to perform verification using the SAT solver due to insufficient memory on the experimental server. This is due to an explosion in the number of logical expressions during program encoding. In addition, when verifying using the SMT solver, an internal error occurred, just as it did in verification2.
• memory-check
  Verification 1 and 2 both showed that there were no memory leaks in the TOPPERS/ASP3 kernel. This demonstrates that a bounded model checker can also verify for memory leaks in an RTOS kernel.
• pointer-check
  NULL pointer accesses were detected in both Verification 1 and 2. This means that tasks in the running state are inserted into the ready queue as ready tasks during task initialization. However, since the ready queue is empty at initialization, a null pointer access is detected when an element is inserted into the ready queue. This demonstrates that a bounded model checker can also verify for null pointer accesses in an RTOS kernel.

6. Summary and Future Plans

In this study, we propose a new verification method that formally verifies the correctness of task scheduling and the priority inversion phenomenon using the bounded model checkers CBMC and ESBMC, and present verification examples. Through experimentation, we were able to verify the correctness of the scheduling process and detect priority inversion using a sample program that had errors intentionally embedded in it. As a result, system designers can use the proposed method to verify the validity of the scheduling in advance and detect priority inversion before it occurs. As a result, this helps prevent various system malfunctions, such as timing errors. Furthermore, experiments showed that the SAT solver was significantly faster than the SMT solver. However, for complex computations, the SAT solver was found to fail verification due to insufficient memory. Therefore, simple problems should be solved using a SAT solver, while complex problems should be solved using an SMT solver.

Moving forward, we intend to conduct verification using actual systems, as this would further demonstrate the effectiveness of the proposed method if we can verify the validity of the scheduling and detect priority inversion phenomena in real-world systems—such as in-vehicle software—rather than just through sample code.