Real-Time Batch Optimization for the Stochastic Container Relocation Problem

1. Introduction

Generally, in the yard of the container terminal, containers are stacked on top of each other in the ground slots, forming stacks. At the same time, a row comprises several stacks placed abreast, constituting a bay, and each yard block is composed of dozens of bays. The containers in a stack adhere to the last-in-first-out (LIFO) principle. If a target container that needs to be retrieved is not the topmost in its stack and is covered by other containers, the blocking containers must be moved to another stack in the same bay first. As a result, the yard crane is required to perform one or more relocation movements. Moving a blocking container, referred to as relocation (rehandling or reshuffling), is an unproductive operation. A high proportion of unproductive operations resulting from container rehandling decreases yard operational efficiency, delays the turnaround time of external trucks, and increases the detention time of vessels. Thus, the container relocation problem (CRP) is a significant source of inefficiency in container terminals and has received considerable attention from terminal operators and scholars.

The CRP aims to find optimal yard crane operation plans for retrieving all the containers in a bay while minimizing the number of relocations. Consequently, the CRP model needs to explore all operation options to find the optimal solution, resulting in an exponential increase in computational effort as the number of containers increases. As an NP-hard problem Caserta et al. [1], it means that it is difficult to obtain an optimal solution for a large-scale instance because it is very time-consuming. Comparatively, getting the approximate optimal solution using heuristic algorithms is valuable and more efficient. Many heuristics, such as ERI ([2]), Chain-LAN ([3]), LA-N ([4]), GLAH ([5]), are outstanding in solving the CRP. However, these heuristics are built based on clear retrieval sequences and may not apply to the relocation problem with uncertain retrieval orders. The main challenge and complexity of the CRP for inbound containers are that it is difficult to accurately master the retrieval order of containers in advance since the arrival time of trucks is random[6]. The uncertainty of the retrieval order of inbound containers makes the CRP for inbound containers intractable.

Stochastic container relocation problem (SCRP) is a CRP variant for inbound containers, also called a CRP with time windows. In the SCRP, the containers are divided into different batches (groups) according to their departure time window, i.e., a batch of containers indicates that the set of containers is stacked in the same bay and with the same retrieval time window. Usually, the retrieval time window is obtained from the truck appointment system (TAS), which has recently been implemented in many container terminals. Containers are ordered so that all containers in a batch must be retrieved before any containers from a later batch. Furthermore, the relative retrieval order of containers within a given batch is assumed to be a random permutation [2,7,8,9]. So, the SCRP is dedicated to retrieving all the containers in a bay while minimizing the expected number of relocations during the pick-up operation process.

Due to the uncertainty of the arrival time of the external trucks and the limited number of yard cranes, the pick-up operation process of inbound containers in a bay can be regarded as a multi-round process. If an external truck (ET) has arrived at the terminal and there is no yard crane to serve the destination bay, the ET must wait until a yard crane is moving into the destination bay to complete pick-up and departure. As shown in Figure 1 (a), we define an operation round as the bay is in busy state, whose period that begins when the yard crane moves into the destination bay and ends when the yard crane moves out the destination bay. Figure 1 (b) tells us that the whole retrieval process of a bay can be divided into three parts — completed operation rounds, real-time operation rounds and future operation rounds. The completed operation rounds represent that the retrieval and relocation operations have been finished, which is irreversible. The real-time operation round refers to when a yard crane provides a loading service for the waiting ETs. The operations within the real-time operation round are unrelated to the completed operation round but will impact future operations.

In this study, a new perspective, namely the Real-time Batch Optimization method, is proposed for dealing with the SCRP during the retrieval process of inbound containers. The new perspective guides us to focus on optimizing the real-time operation round and constructing the evaluation function for evaluating the expected increment of the number of blocking containers with the bay layout change instead of exploring the subsequent operation processes to improve the solution efficiency. In the new approach, we first categorize the ETs into batches (groups) based on their reservation information from the TAS, as in the existing literature. Of course, the retrieval sequence of the ETs belonging to different batches is precise, and the retrieval sequence of the ETs within the same batch is still being determined. Then, based on this grouping, all the arrived ETs within the real-time operation round are defined as a specific batch (group) whose retrieval priories are set to zero, representing the highest pick-up priority. Correspondingly, the target containers within the real-time operation round are a batch with a retrieval priority of 0, which may only be part of the containers booked in the current time window. Moreover, their pick-up service order of all arrived ETs is adjustable (no longer following the first-come-first-served principle), and the optimal pick-up order is determined based on the bay layout to minimize the number of relocations.

The contributions of this paper are as follows:

• We derive an improved dynamic programming (DP) model for the real-time retrieval operation scenarios of SCRP. Compared to the general SCRP, this improved DP model focuses on optimizing the real-time operation round and less explores the future operation processes.
• We propose a new heuristic relocation strategy, namely Least-relocations-and-Lowest-precedence-gap (LL), which learns from the expected relocation index (ERI) heuristic presented by [2] and the expected min-max (EM) heuristic proposed by [9].
• We introduce two novel relocation rules: moving ahead for a sequential-placed slot (MSS) and freeing up for a sequential-placed slot (FSS). The former rule assigns, as far as possible, a sequential placement for the potential blocking containers in the future in advance. The latter rule tries to find a stack that can free its topmost container to satisfy a sequential placement for the current blocking container.
• A novel heuristic algorithm, called the sequential-placement first heuristic ($SPFH$), is devised. It takes advantage of the properties of the two relocation rules mentioned above. The algorithm is explained with pseudo-code in Algorithm 3.

The rest of the paper is structured as follows: In Section 2, we quickly survey recent literature on the CRP. Section 3 gives a detailed description of the real-time CRP and the formulation of the improved DP model. Section 4 describes the proposed $SPFH$ algorithm in detail. We report the results of our computational experiments with the methods discussed in this paper in Section 5. Finally, Section 6 concludes the study and suggests future research directions.

2. Literature Review

Two fields have been extensively investigated in the existing literature to reduce relocations during the retrieval process of inbound containers. In the first field, the objective is to allocate stacking locations for new unloaded containers to reduce relocation moves in the future. Such as, the formulations for evaluating the expected number of rehandles were proposed in [10,11] based on the segregation storage strategies. Other similar studies on generating container storage strategies can be found in [12,13,14]. The second field, also known as the container relocation problem, focuses on the relocation movements during the retrieval process and is aimed at deciding to which stack the blocking containers should be moved to be considered to reduce the relocations.

Due to the containers stacked on top of each other and the limited yard area, the relocating operation is unavoidable in the container terminal yard. So, the CRP has received much attention from scholars for decades. Various integer programming (IP) or mixed-integer programming (MIP) models [see, e.g., [15,16,17,18,19] and exact algorithms based on tree search procedures have been developed, such as tree search algorithms [20,21,22], beam search algorithms [23], (iterative deepening) A* algorithms [24,25], B&B algorithms [19,21,26,27,28,29,30,31], and abstract approaches [2]. However, the CRP is an NP-hard problem. For larger-scale instances, the exact algorithms or models mentioned above are time-consuming and impractical for real-time operation demands. Therefore, researchers have been increasingly exploring heuristic algorithms to overcome the computational complexity of CRP. So far, many outstanding heuristic algorithms have been developed, such as beam search algorithms [23,28,29,31], greedy heuristics [3,5,32], and other heuristics based on relocation rules [2,8,9,21].

In the earlier literature, due to the difficulty of obtaining the retrieval time information in advance, some studies constructed random models for CRP based on assuming that the retrieval processes of the inbound containers conform to some probability distribution [15,33,34]. Moreover, some papers proposed the online CRP models, where the retrieval orders of the containers are revealed progressively with the arrival of ETs [2,7,8]. Truck appointment systems (TAS) allow terminal operators to obtain the containers’ retrieval time window through the TAS. Based on the appointment information, some researchers proposed batch model where the containers within the same appointment time windows are considered a batch. Based on the batch model, [9] proposed the stochastic container relocation problem (SCRP), where the retrieval order between different batches is clear but uncertain within the same batch. It means the containers within the same batch have the same retrieval probability. Indeed, the retrieval sequence of the same batch containers is indistinguishable until associated ETs arrive.

For the CRP, relocating operations occur when the container pick-up orders do not match their stacking sequence. Many studies address the container storage allocation/re-allocation problem, but only some focus on controlling the retrieval sequence. [7] first proposed an embryo of the Batch Optimization method, in which a batch of ETs has arrived, and their pick-up service orders are adjustable. Nevertheless, they only apply this method to the first batch to avoid delaying the other ETs’ waiting time. [?] proposed a rehandling model that considered adjustments to the container pick-up order within each group to minimize the number of rehandles. They grouped the ETs according to the appointment time slots in the TAS. [6] proposed a flexible service policy for the SCRP, which assumes all the ETs corresponding to the containers of each batch arrived in advance and dictates the retrieval orders of the current batch’s inbound containers to minimize the number of relocations. In practice, some arriving ETs within a real-time operational round are likely different from their reservations. Therefore, the abovementioned studies need more practicality and flexibility in real-time operation scenarios. However, they show us a novel optimization perspective on the CRP. So far, there is little literature discussing the novel perspective. Summarily, the batch optimization method has two important features—certainty (the arrival of ETs means that the corresponding target containers of a batch have been identified) and flexibility (the pick-up order of the containers of a batch is adjustable).

The novelty of this work. Compared with the previous studies, our paper proposes a new perspective on studying the real-time operation scenario for the SCRP. In our work, we formulate a new dynamic programming model and develop a novel heuristic algorithm, which performs better on the runtime and solution quality than existing outstanding heuristics.

3. Problem description and formulation

3.1. Retrieval process analysis

Due to the limited number of yard cranes, each must move back and forth between different bays rather than being dedicated to a specific unit. When a truck arrives, the yard crane may not serve the destination bay immediately. In such cases, the ETs must wait until a yard crane enters the destination area. As shown in Figure 1, the operation state of a bay changes as the yard crane moves in or out. When the yard crane moves in the bay, its operation state changes into busy, i.e., the bay begins its real-time operation round. When the yard crane moves out of the bay, it implies that the real-time operation round has finished and changed into the completed operation round. Meanwhile, the bay state changes into idle.

Consider a bay with N containers piled on W stacks, and the height of each stack cannot exceed H tiers. In the initial state, all the N containers in the bay have been appointed their departure time windows according to the corresponding ETs’ reservation arrival period. In turn, they can be classified into G groups based on their respective booking departure times and numbered $1,\dots ,G$, which implied their retrieval priorities (the smaller group index, the higher retrieval priority).

Suppose this instance requires K ($K\ge 1$) operation rounds to finish retrieving all the N containers. Let $S^k$ denote the bay layout after the completion of kth operation round, especially indicating the initial layout when $k=0$. Suppose we have completed the first $k-1$ operational rounds, and then we need to complete the kth operation round. We aim to find an optimized solution $a^k$ for the kth operation round that minimizes the number of relocations during the kth operation round. Once the kth operation round finishes, the bay layout will be transformed from $S^{k-1}$ to $S^k$, which becomes the basis for $(k+1)$th operation round. The change of the bay layout between two adjacent operation rounds is described by the following equation 1.

$$S^{k-1}\stackrel{a^k}{\to }{S}^k,1\le k\le K$$

(1)

The $a^k$ is a set of operation instructions. If $a^k$ includes some relocation operation instructions, some containers will be moved from their stacks into other new stacks. For the restricted CRP, the relocated containers are only the blocking containers above the targets. The relocated containers may contain containers other than the blocking containers above the targets for the unrestricted CRP. However, these movements may result in the addition of the number of blocking containers for the moved-in stacks and impact the $(k+1)$th and subsequent operation rounds. Our objective is to minimize the addition of the number of blocking containers in the real-time operation round.

Moreover, during real-time operations, the retrieval sequence of target containers and the movement orders of blocking containers determine the changes in the bay layout, and different bay layouts will produce different increments in the number of blocking containers and other impacts on the subsequent operation process. In the real-time operation round, when multiple target containers need to be picked up, it makes sense to choose a reasonable target container pick-up order that can produce fewer relocation operations, even though it may prolong the waiting time of some ETs. Therefore, the method proposed in this paper allows for adjusting the pick-up order according to the bay layout, integrating the optimization of the pick-up sequence and the reallocations of blocking containers to minimize the increment of the expected number of blocking containers (ENBC). No doubt, the pick-up order in practice is likely to be inconsistent with the ETs’ arrival sequences, which will result in loading operations being advanced for some ETs and delayed for others. In practice, the number of waiting ETs for retrievals is usually small, and as the number of relocations decreases, the whole operation time decreases, and the delays incurred by individual ETs are acceptable.

3.2. Formulation for the real-time scenario of CRP

Learning from [2], we first present a simplified DP model of real-time CRP. Let $R\left(S^k\right)$ denote the minimum number of relocations required to remove all the remaining containers based on the layout $S^k$, and let $q\left(a^k\right|S^{k-1})$ denote the number of realized operations that occur during the kth operation round. The DP model can be formulated as a recursive function as in Eq. (2).

$$\left\{\begin{array}{c}R\left(S^{k-1}\right)=\underset{a^k}{\min }\left\{q\left(a^k\right|S^{k-1})+R\left(S^k\right)\right\},1\le k\le K\hfill \\ R\left(S^K\right)=0\hfill \end{array}\right.$$

(2)

Another equivalent form of Eq. (2) is shown in Eq. (3)

$$R\left(S^{k-1}\right)=\underset{a^k}{min}\left\{q\left(a^k\right|S^{k-1})+\underset{a^{k+1}}{min}\left\{q\left(a^{k+1}\right|S^k)+\cdots +\underset{a^K}{min}\left\{q\left(a^K\right|S^{K-1})+\right\}\cdots \right\}\right\}$$

(3)

From Eq. (3), to obtain the optimal operation plan $a^k$ for the kth operation round, the solution process requires simultaneous evaluation of the optimal operation plan from the $(k+1)$th operation round to the last operation round. So, solving the total optimization objective $R\left(S^0\right)$, which is the minimum number of relocations for picking up all containers during the whole retrieval process, is difficult because the complete solving process will generate a large number of redundant computation efforts. Consider it another way, and we can try to find an approximate optimal solution for it.

According to the Eq. (2), if $a^{*}$ is the theoretically optimal operation plan for the kth round, then we have $R\left(S^{k-1}\right)=q\left(a^{*}\right|S^{k-1})+R\left(S^k\right)$. Since solving for the theoretical optimum is difficult, assuming that we get an approximate optimal solution $a^k$ for the kth operation round, we can derive an inequality Eq. (4).

$$R\left(S^{k-1}\right)\le q\left(a^k\right|S^{k-1})+R\left(S^k\right)$$

(4)

Without loss of generality, if $a^1,a^2,\cdots ,a^K$ are approximate optimal solutions for each round, respectively, then we must have inequality Eq. (5).

$$R\left(S^0\right)\le \sum _{k=1}^{K}q\left(a^k\right|S^{k-1})$$

(5)

In this way, we convert the solving process of Eq. (3) to summing the number of practical relocations during each operation round. We can ensure the desired suboptimal solution if we minimize the number of relocations within each round and the impact on the subsequent operation rounds.

According to the definition, $q\left(a^k\right|S^{k-1})$ includes two aspects: forced relocations and relocations to free up storage placements. The forced relocations result from blocking the target containers during the real-time operation round. The relocations to free up storage placements result from the containers having nothing to do with the target containers and aim to free up good placements for the blocking containers. In addition, some blocking containers may be relocated twice or more due to all candidate storage positions containing the target containers, but this case is included in the forced relocations. Therefore, we can derive the following inequation as Eq. 6.

$$R\left(S^0\right)\le \sum _{i=1}^K\left(\left|\bigcup _{c\in P_k}{B}_c^k\right|+\left|U_k\right|\right)$$

(6)

$B_c^k$ denotes the blocking containers of the target container c in kth operation round, and $U_k$ means the set of relocated containers not within the same stack as the target containers during the kth operation round.

Let $LB\left(S^k\right)$ denote the number of blocking containers in the layout $S^k$, which will require rehandling operations in the remaining rounds. $P_k$ indicates the set of target containers in kth operation round. During the kth operation round, Each container $b\in B_c^k$ must be moved into other stacks and may become new blocking containers in the move-in stack. Let $f(b,s)$ denote the expected additional number of the blocking containers when the container b is moved into the stack s. Its formula will be given later. Thus, after completing the kth operation round, the number of blocking containers in the layout $S^k$ is as Eq. 7.

$$LB\left(S^k\right)=LB\left(S^{k-1}\right)-\left|\bigcup _{c\in P_k}{B}_c^k\right|+\sum _{s=1}^W\sum _{b\in \mathcal{B}}f(b,s),\mathcal{B}=\left\{\bigcup _{c\in P_k}{B}_c^k\right\}\cup U_k$$

(7)

When we have removed all the containers, the last bay layout $S^K$ is empty and $LB\left(S^K\right)=0$. Iterative summation of Eq. 7 for k from 1 to K yields Eq. 8.

$$\sum _{k=1}^K\left|\bigcup _{c\in P_k}{B}_c^k\right|=LB\left(S^0\right)+\sum _{k=1}^K\sum _{s=1}^W\sum _{b\in \mathcal{B}}f(b,s),\mathcal{B}=\left\{\bigcup _{c\in P_k}{B}_c^k\right\}\cup U_k$$

(8)

The total optimizing objective of this study is $minR\left(S^0\right)$. It satisfies inequality Eq.9.

$$R\left(S^0\right)\le LB\left(S^0\right)+\sum _{k=1}^K\sum _{s=1}^W\sum _{b\in \mathcal{B}}f(b,s)+\sum _{k=1}^K\left(\left|U_k\right|+\left|{ϵ}_k\right|\right),\mathcal{B}=\left\{\bigcup _{c\in P_k}{B}_c^k\right\}\cup U_k$$

(9)

The initial layout of a bay is fixed, and the $LB\left(S^0\right)$ can be considered a constant. According to Eq. (9), a suboptimal solution to the CRP can be obtained by minimizing the expected increment in the number of blocking containers and the number of rehandles due to relocations within each operating round. Therefore, we concentrate on optimizing the real-time operation process of each round and decrease the computational efforts for assessing the influence of relocating obstructive containers within each round on subsequent operation rounds. This method will significantly enhance the solution efficiency of the CRP and make it suitable for real-time scenarios.

4. Algorithm design

According to Eq. (9), the real-time scenario of the CRP focuses on two aspects: (1) Minimizing the instant relocation number during the real-time operation round; (2) Minimizing the incremental number of the blocking containers after each operation round.

4.1. Preliminaries

Before describing the implementation of the algorithms in detail, we first define the placement state of containers and then introduce three heuristic relocation rules.

4.1.1. Placement state of containers

Let $p\left(c\right)$ represent the retrieval priority value of container c and $p_{min}\left(s\right)$ denote the minimum retrieval priority value of the container set s (If s is a stack number, it represent all the containers in stack s), the smaller value, the higher priority. Specifically, $p_{min}\left(s\right)=N+1$ if s is an empty set or stack. Let $under\left(c\right)$ denote the containers set within the same stack with container c and below it.

We define three placement state types of containers as follows. The placement state of the container c is to be known as: (1) sequential-placement if $p\left(c\right)<p_{min}\left(under\left(c\right)\right)$ is satisfied; (2) inverted-placement if $p\left(c\right)>p_{min}\left(under\left(c\right)\right)$ is satisfied; (3) level-placement if $p\left(c\right)=p_{min}\left(under\left(c\right)\right)$ is True. A sequential-placement container means that all other containers placed below have lower retrieval priorities than it. Similarly, an inverted-placement container means that all other containers placed below have higher retrieval priorities than it, i.e., it has a bigger retrieval priority value than the other containers below it. A level-placement container has the same retrieval priority as the highest retrieval priority of the other containers placed below.

4.1.2. The evaluation method of ENBC

Based on their predetermined precedence (e.g., appointment time window), the containers with higher precedence will be retrieved earlier than those with lower precedence during retrievals. Therefore, a sequential-placement container will be picked up earlier than other containers below it without relocation. Conversely, an inverted-placement container must be relocated at least once, as some containers below it will be picked up earlier. We assume the containers with the same retrieval priority have the same retrieval probability. So, a level-placement container causes a relocation when the same retrieval priority container below it is picked up earlier. The blocking containers in a bay layout include the inverted-placement and level-placement containers, and the ENBC is equal to the sum of the number of inverted-placement containers and the expected relocation number of the level-placement containers. The calculation method for the ENBC can be described as Eq. (10), which has been investigated in [9]. In Eq. (10), $I\left(x\right)$ is an indicator function that equals one if x is True and zero otherwise, and $h\left(r\right)$ indicates the container number within stack r.

$$lb\left(r\right)=h\left(r\right)-\sum _{i=1}^{h\left(r\right)}\left(\frac{I\left(p\left(c_i\right)=\underset{1\le j\le i}{\min }\left\{p\left(c_j\right)\right\}\right)}{{\displaystyle \sum _{j=1}^i}I(p\left(c_i\right)=p\left(c_j\right))}\right)$$

(10)

There is no increase in the number of blocking containers in a candidate stack when a blocking container becomes a sequential-placement container after being moved into the candidate stack during retrieval. We refer to the candidate stacks for the blocking container that satisfy the above condition as sequential-stack (SS for short). Contrarily, the ENBC of the candidate stack will increase when a blocking container becomes the inverted-placement or level-placement container after being moved into a candidate stack. The candidate stack that satisfies the condition is referred to as inverted-stack (IS for short) or level-stack (LS for short), respectively. The notation $f(c,r)$ has been defined previously, which denotes the increment of ENBC of the stack r after relocating the container c into stack r, and the formula is shown Eq. (11).

$$f(c,r)=\left\{\begin{array}{cc}0,& r\in SSs,\\ 1,& r\in ISs,\\ \frac{m-1}{m},& r\in LSs,\end{array}\right.$$

(11)

Where $m={\displaystyle \sum _{j=1}^{h\left(r\right)}}I(p\left(c\right)=p\left(c_j\right))$ denotes the number of containers with the same priority as container c in the stack r.

4.1.3. Heuristic relocation rules

We introduce three relocation rules in this subsection. The first relocation rule, namely the Least-relocations-and-Lowest-precedence-gap (LL) rule, draws inspiration from relocation rules such as ERI [2], Minmax [35], and EM [9]. The other two relocation rules are used to compensate for the deficiencies of the LL rule, including the prospective re-allocations for some non-instant-relocated blocking containers to produce a better layout.

LL rule 

Two conditions construct the LL rule. They are first, selecting the set of stacks that produces the smallest increment of the ENBC after reshuffles—second, choosing the move-in stack from the set of stacks obtained in the “first step” with the closest priority value for the blocking container. Eq. (12) describes the mathematical formula of the LL rule.

$$s^{*}=\underset{s}{argmin}\left\{\left|p_{min}\left(s\right)-p\left(c\right)\right|\right\},s\in \left\{r\left|minf(c,r)\right.\right\},s\ne s\left(c\right)$$

(12)

$s\left(c\right)$ denotes the function to get the stack index where the container c is stacked. If $f(c,s^{*})=0$, it indicates that the topmost slot of stack $s^{*}$ is a sequential stack for blocking container c. Therefore, moving container c to stack $s^{*}$ does not increase the ENBC. If $f(c,s^{*})\ne 0$, it indicates that there is no sequential stack for blocking container c to move in. The LL rule provides a method for selecting the move-in stack for container c, which suggests that the stack with the closest precedence to container c is the best choice. This rule has the advantage of keeping the stack with lower priority to provide sequential stacks for subsequent blocking containers as much as possible. Figure 2 illustrates an example of the LL rule.

The $LL$ rule has two potential areas for improvement. Firstly, it is a greedy approach that always assigns a sequential placement for the relocating container whenever possible, which will likely trap into the local optimum. In other words, reallocating a blocking container with higher retrieval priority to a sequential placement with lower priority may result in losing sequential-placed slots for subsequent relocations. Secondly, when only inverted-placement slots are available for the blocking container, the subsequent blocking containers likely have no more sequential placements to select from, which will directly make additional relocations or result in more potential relocations in the following retrieval processes. The next two relocation rules remedy the deficiencies of the $LL$ rule in the previously mentioned aspects.

Moving ahead for Sequential Stack (MSS) 

Given the blocking container c that is next to be relocated, its optimal reallocation stack $s^{*}$ is determined according to the $LL$ rule. If stack $s^{*}$ is sequential-stack for blocking container c, we try to find another inverted-placement container, which is the topmost item in its stack s ($s\ne s^{*}$) and denotes as $top\left(s\right)$. If stack $s^{*}$ is sequential-stack for container $top\left(s\right)$ and moving it into stack $s^{*}$ before container c does not change the sequential-placement state of container c. We define this method as a relocating policy, known as the Moving-ahead for Sequential Stack (MSS) rule. The MSS rule aims to move a inverted-placement container into a sequential-stack stack in advance and reduce the likelihood of the container $top\left(s\right)$ becoming a blocking container again in the subsequent operation process. This rule needs to comply with the following conditions: (1) There are at least two empty slots in stack $s^{*}$; (2) The topmost container $top\left(s\right)$ in stack s must be in inverted-placement state, i.e., its retrieval priority satisfies the condition $p\left(top\left(s\right)\right)>p_{min}\left(s\right)$. (3) the retrieval priority of container $top\left(s\right)$ must satisfy the condition $p\left(c\right)<p\left(top\left(s\right)\right)<p_{min}\left(s^{*}\right)$, i.e., the stack $s^{*}$ is sequential-stack for containers c and $top\left(s\right)$ and the retrieval priority of container $top\left(s\right)$ is lower than container c. Figure 3 illustrates the processing procedure of the MSS rule.

Freeing up for Sequential Stack (FSS) 

For a blocking container c, if only inverted-stack stacks are available, directly moving container c into one of the stacks will add a new blocking container and, in turn, add a relocation during the subsequent retrieval processes. In this case, selecting a suitable stack s and moving its topmost container $top\left(s\right)$ to another stack $s^{\prime }$ will free up an sequential-placement for container c. If stack $s^{\prime }$ is a sequential-stack for container $top\left(s\right)$, the total operation cost of the “free up” operations is the same as that of direct moving. As well as it may also increase the possibility of selecting sequential-placement for the subsequent relocations. We refer to the above method as a new relocation policy: freeing up for the sequential stack (FSS) rule. The FSS rule needs to comply with the following conditions: (1) The available reallocation positions for the blocking container c are inverted-placements; (2) If stack s is a candidate sequential-placement for the blocking container c by freeing up the topmost item $c^{\prime }$, while the retrieval priority value satisfies $p\left(c^{\prime }\right)<p\left(c\right)<p\left(under\left(c^{\prime }\right)\right)$. (3) The container $c^{\prime }$ will be moved into stack $s^{\prime }$, which must be the sequential-stack for container $c^{\prime }$.

We now continue to analyze the instance depicted in Figure 4. In the final configuration, container $(1,2)$ is the current forced relocating item, and two available stacks, the second and fourth stack, are both inverted stacks. Let us move container $(4,1)$ from the fourth to the second stack and obtain a sequential stack by consuming one practical relocation movement. Furthermore, this reshuffle is cost-effective since it may provide an additional sequential-placement in subsequent processes. Figure 4 illustrates the detailed processing of the FSS rule.

4.2. sequential-placement first heuristic ($SPFH$)

This section presents the $SPFH$ algorithm for the real-time operation round of SCRP. The algorithm structure of $SPFH$ includes two levels. The outer level adopts a policy iteration mechanism, i.e., the set of policies comprising all possible pickup sequences, from which the best operation solution has the least cost; the inner level is the specific process of simulating the operation solution and evaluating the cost for each pickup sequence.

4.2.1. The Outer Level of $SPFH$

$SPFH$ takes two inputs: $S_0$ denotes the initial configuration of the current operation round of the bay, and P denotes the target container set of the current operation round. In the outer level of $SPFH$, the procedure, namely AutoRetrieval, is a conventional prioritization mechanism that checks and automatically picks up the target containers if retrievable before the start of each operation round or after each relocation. This mechanism makes the configuration irreducible while freeing up optional slots for other blocking containers. The procedure, namely GenerateSequence, generates all possible pickup orders for the target containers of the current operation round. Then, iterate these orders to obtain the best operation plan for the yard cranes with minimal operation costs. The procedure, namely Simulation-Evaluation, is the inner level of the $SPFH$ algorithm. The pseudo-code of the outer level of $SPFH$ is shown in Algorithm 1, and the pseudo-codes of the procedure AutoRetrieval and the procedure GenerateSequence are shown in Algorithm 2.

Algorithm 1 Outer level: the main procedure of $SPFH$ algorithm

1: procedure SPFH($S_0$,P)     2:     Initialize $S_{cur}\leftarrow S_0$, $so{l}_{cur}\leftarrow ⌀$, $so{l}_{best}\leftarrow ⌀$, $F_{best}=\infty$     3:     AutoRetrieval($S_{cur}$,$so{l}_{cur}$,P)    4:     if P is empty then $F_{best}\leftarrow 0$, $so{l}_{best}\leftarrow so{l}_{cur}$     5:     else    6:         $Q\leftarrow$GenerateSequence($S_{cur}$,P)     7:         while Q is not empty do     8:            $q\leftarrow$ select first item from the pickup sequence set Q     9:            $F_{cur}\leftarrow$Simulation-Evaluation(q,$S_{cur}$,$so{l}_{cur}$)    10:            if $F_{cur}<F_{best}$ then    $F_{best}\leftarrow F_{cur}$, $so{l}_{best}\leftarrow so{l}_{cur}$    11:            else if $F_{cur}=F_{best}$ then $so{l}_{best}\leftarrow so{l}_{best}\cup so{l}_{cur}$    12:            $Q\leftarrow Q\setminus \left\{q\right\}$    13:     return $F_{best}$, $so{l}_{best}$

Algorithm 2 The procedures: AutoRetrieval and GenerateSequence

1: procedure AutoRetrieval(S,$sol$,T)     2:     for all $t\in T$ do     3:         if t is topmost item in stack $w\left(t\right)$ then     4:            $S\leftarrow S\oplus 〈t,w\left(t\right),0〉$, $sol\leftarrow sol+〈t,w\left(t\right),0〉$, $T\leftarrow T\setminus \left\{t\right\}$     5: procedure GenerateSequence(P)     6:     $W\leftarrow$Get the set of stacks containing the target containers in P.     7:     $to{p}_1\left[W\right]\leftarrow$Get the top target container in each stack of W and remove them from corresponding stack.     8:     $Q_1\leftarrow$Generate the set of full permutations from each target container in $to{p}_1$.     9:     $Q_2\leftarrow$    10:     $to{p}_2\left[W\right]\leftarrow$Get the top target container in each stack of W and remove them from corresponding stack.    11:     while An item in the $to{p}_2\left[W\right]$ is not empty do    12:         for all $q_1\in Q_1$ do    13:            for all $t_1\in to{p}_1$ do    14:                $l_1\leftarrow$Finding the position index of $t_1$ in $q_1$    15:                for all $t_2\in to{p}_2$ do    16:                    for all $i\in [l_1,len\left(q_1\right)]$ do    17:                        $q_2\leftarrow$Insert $t_2$ sequentially into $q_1$ at ith item, and generate a new sequence.    18:                        $Q_2\leftarrow Q_2\cup \left\{q_2\right\}$    19:         $Q_1\leftarrow$, $Q_1\leftarrow Q_2$, $Q_2\leftarrow$, $to{p}_1\leftarrow to{p}_2$    20:         $to{p}_2\left[W\right]\leftarrow$Get the top target container in each stack of W and remove them from corresponding stack.    21:     return $Q_1$

4.2.2. The Inner Level of SPFH

The objective of the inner level of SPFH is to simulate the relocation operation processes and estimate the associated operation costs concerning a given pickup sequence. This procedure involves removing the target containers of the current operation round and moving their blocking containers into the best placements with the least cost (the sum of the number of instant relocation moves and the addition of ENBC).

The detailed simulation and evaluation procedure consists of the following steps. Firstly, we apply the LL rule to identify the optimal stack $s^{*}$, which will minimize the addition of ENBC among all assignable slots to which we can move the blocking container directly. However, the earlier reallocations of the blocking containers occupy reasonable placements and may lead subsequent blocking containers to lose the well-stacking slots. Hence, we will consider the impact of current reallocation decisions on subsequent blocking containers and determine whether there are better solutions than the current selection. Suppose the selected stack $s^{*}$ is a sequential-stack for the current blocking container. In this case, we will check for some inverted-placement containers that satisfy the MSS rule to increase the chances of choosing a well-stacking position for other blocking containers. Similarly, suppose the selected stack $s^{*}$ is a inverted-stack, we will try to find a stack $s^{\prime }$ satisfying the FSS rule that the stack $s^{\prime }$ is a sequential-stack for the current blocking container after moving the topmost container in stack $s^{\prime }$ to another stack. The optimal result is that the non-forced relocating container obtains a new well-stacking placement. Otherwise, relocating the non-forced container will increase the final operational cost in the future. Moreover, when stack $s^{*}$ provides a level-placement for the current relocating container, neither the MSS rule nor the FSS rule is applicable because they may exacerbate the problem. Algorithm 3 presents the pseudo-code for the inner level of $SPFH$.

Algorithm 3 Inner level: Policy iteration procedure

1: procedure Simulation-Evaluation(q, S, $sol$)     2:     $P\leftarrow q$,$S_{cur}\leftarrow S$;     3:     while P is not empty do     4:         $p\leftarrow$ get the first item of P     5:         $B_p\leftarrow$ get the blocking containers of the target container p     6:         for all $b\in B_p$ do     7:            $opr\leftarrow$, $s^{*}\leftarrow LL(b,S_{cur})$           ▹ Get the stack for container b according to LL rule     8:            if $f(b,w\left(b\right),s^{*})=0$ then     9:                $s^{\prime }\leftarrow MSS(s^{*},b,S_{cur})$           ▹ Find a stack in satisfying MSS rule    10:                if $s^{\prime }$ is not NULL then 11:                    $opr\leftarrow opr+\langle top\left(s^{\prime }\right),s^{\prime },s^{*}\rangle$, $opr\leftarrow opr+\langle b,w\left(b\right),s^{*}\rangle$    12:            else if $f(b,w\left(b\right),s^{*})>0$ then    13:                $s^{\prime }\leftarrow FSS(s^{*},b,S_{cur})$           ▹ Find a stack in satisfying FSS rule    14:                if $s^{\prime }$ is not NULL then    15:                    $s\prime \prime \leftarrow LL(top\left(s^{\prime }\right),S_{cur})$, $opr\leftarrow opr+\langle top\left(s^{\prime }\right),s^{\prime },s\prime \prime \rangle$, $opr\leftarrow opr+\langle b,w\left(b\right),s^{\prime }\rangle$    16:                else   17:                    $opr\leftarrow opr+\langle b,w\left(b\right),s^{*}\rangle$    18:            else   19:                $opr\leftarrow opr+\langle b,w\left(b\right),s^{*}\rangle$    20:            $S_{cur}\leftarrow S_{cur}\oplus opr$, $sol\leftarrow sol+opr$           ▹⊕ is an operator that indicates that $opr$ will be executed based on the bay layout $S_{cur}$ and generate a new layout.    21:            AutoRetrieval($S_{cur}$,$sol$,P)    22:     return $∥opr∥$

5. Computational Experiments

In this section, we present some computational experiments to validate the effectiveness of the $LL$ heuristic and $SPFH$ algorithm for the real-time operations of SCRP.

5.1. Implementation details

Our experiments are implemented by Matlab 2016b and conducted on a PC with an Intel Core I7-9700K CPU at 3.6GHz and 16GB RAM. The program code, results and instances used in this section are available at https://github.com/zhou-sf/rt-scrp. We generate our experiment dataset based on the existing dataset from [9] and complete our experiments from four cases. Each case of the experiments is implemented with 30 instances. We consider the $LL$ heuristic as a standalone algorithm for comparison experiments.

Our experiments validate the effectiveness and performance of the LL heuristic and $SPFH$ algorithm under small-batch and large-batch instances, respectively. In the case of small-batch instances, the configuration sizes of the instances vary from $\mathbf{S}=\mathbf{5},\dots ,\mathbf{10}$ stacks and $\mathbf{T}=\mathbf{3},\dots ,\mathbf{6}$ tiers and batch sizes of each operation round vary from $\mathbf{B}=\mathbf{1},\dots ,\mathbf{4}$ containers. The configuration sizes of the large-batch instances vary from $\mathbf{S}=\mathbf{10},\mathbf{12}$ stacks and $\mathbf{T}=\mathbf{8},\mathbf{9},\mathbf{10}$ tiers, and batch sizes of each operation round vary from $\mathbf{B}=\mathbf{5},\dots ,\mathbf{12}$ containers.

5.2. Experiment results

5.2.1. Effectiveness validation under small-batch instances

36 configurations, each of 30 instances, are solved using the proposed heuristic $LL$ and algorithm $SPFH$. Each configuration has $S\times T\times FillRate$ containers, in which S is the number of stacks of the bay, T is the maximum number of tiers in a stack, and the $FillRate$ is the occupancy rate of the slots in the bay. In this experiment, the value of S varies between 5 and 10, and the T varies between 3 and 6. The $FillRate$ is divided into two cases: 50% and 67%.

The experiments aim to validate the effectiveness of our algorithms. Usually, many container terminals stack their containers in not more than six tiers, and there are rare cases in which the number of target containers within a bay during an operation round is more than 5. Suppose our algorithms perform well regarding runtime and optimization objectives under small-batch instances. In that case, our algorithm can be applied to most container terminals.

(1) Comparison of complete time

Table 1 and Table 2 summarize the experiment’s results of Case 1 with two storage space occupancy rates, 50% and 67%. The rows labeled as “C” indicate the container number of each configuration, and the rows labeled as “Solved” show us the number of instances solved in the form $x/30$. We can find that all 30 instances for each configuration are solved optimally by the algorithm $SPFH$ and heuristic $LL$, respectively. The solution time in the two tables refers to the runtime of the most time-consuming instance of each configuration. The running time of the algorithms is related to the number, the location of target containers per batch, and the bay layout. Since we introduce the flexible pick-up order mechanism into our algorithms, iterating through all possible pick-up sequences may result in a more time-consuming solution process. The worst case in Table 1 is that an instance with configuration (5 stacks and three layers) and with 67% fill rate has four target containers, and it takes about 0.134 seconds to solve, which exceeds ten times that of the solution time of the other instances. In this case, the solution time of most instances is less than 0.02s.

(2) Analysis of the results

Table 3 and Table 4 show us the experiment results of the $LL$ heuristic and $SPFH$ algorithm under 50% and 67% fill rate, respectively. In the two tables, the notation “$IB$” refers to the initial number of blocking containers for each instance. Its value equals the sum of the number of forced relocations and potential relocations of a bay’s initial layout, determined by the number of the inverted-placement and level-placement containers, respectively. Considering that the pickup order is adjustable during the realized operations, some potential relocations (determined by level-placement containers) may not occur in the realized operations. So, the actual number of relocations ($Act$ for short) generated during practical operations may be smaller than the theoretical value ($IB+I_{eb}$). The $I_{eb}$ indicates the sum of the increment in the expected number of new additional blocking containers and the number of movements other than the forced relocations during all operation rounds of a bay.

Columns $IB$ and $I_{eb}$ indicate each configuration’s average values of 30 instances. The $IB$ is nearly half the sum of the maximum and minimum values of the initial expected number of blocking containers in a bay. In addition, the value of $IB$ increased gradually with the size of the instance configurations. However, its standard deviation $Stdev$ does not reflect a clear pattern, indicating a high degree of randomness in the data of the instances. In other words, it has a high degree of representativeness in the data of the instances in this study. The $I_{eb}$ values of $LL$ heuristic are significantly different from those of the $SPFH$ algorithm due to the differences between the two methods. The $LL$ heuristic only employs the flexible pick-up order policy, whereas the $SPFH$ algorithm incorporates, in addition to this, two relocation rules—$MSS$ rule and $FSS$ rule. So, the $I_{eb}$ values of the $SPFH$ algorithm contain many relocations generated by the two relocation rules, which would be required in subsequent rounds and were executed in advance by the two relocation rules.

Table 5 shows each configuration’s actual number of relocations ($Act$), obtained by simulating the optimal operation plan for each round. According to Eq. 8, the value of $Act$ is smaller than the value of $(IB+I_{eb})$ and is closer to the optimal solution. Column G denotes the error rate between $Act$ and $IB+I_{eb}$ (the value equals $(IB+I_{eb}-Act)/(IB+I_{eb})$). We find a negative value in column G of the $LL$ heuristic at a fill rate of 50%, which indicates that the value of $Act$ is more than $IB+I_{eb}$, the theoretical upper bound of the CRP model. Similarly, more such cases appear in column G of the $LL$ heuristic at a fill rate of 67%. The reason for this is because the $LL$ heuristic, when selecting a position for the blocking container, will prioritize a level stack in the absence of sequential candidate stacks (selecting level stack produces an increment of the expected number of potential blocking container less than 1). However, some potential blocking containers turn out to be real during subsequent operations (producing one realized relocation), so there are more realized relocations than the theoretical optimal operation solution. However, this case does not appear in column G of the $SPFH$ algorithm because the $SPFH$ algorithm reduces the selection of level stacks through both relocation rules — $MSS$ and $FSS$ so that the performance of the theoretical upper bound of the $SPFH$ algorithm meets the anticipative optimization objective. In terms of the values of $I_{eb}$, they are much larger in the $SPFH$ algorithm than in the $LL$ heuristic because the $MSS$ and $FSS$ rules perform some relocations in advance or freeing up the sequential stacks for the blocking containers, which in fact would cause relocations inevitably during the subsequent operations. These advance relocations are counted in $Act$ and $I_{eb}$. Therefore, the $SPFH$ algorithm generally shows a stable optimization effect regarding the number of actual relocations.

(3) Comparison between $LL$ heuristic and $SPFH$ algorithm

As shown in Figure 5 (a) and (c), the $Act$ values of $SPFH$ and $LL$ are very close to each other and are even identical when the number of stack tiers is equal to three. Figure 5 (b) and (d) demonstrate the error of the two algorithms at fill rate 50% and 67%, respectively, and the results of the $SPFH$ algorithm perform better than that of the $LL$ heuristic. Meanwhile, we find that the $Act$ values increase with the number of stacks and the number of stack tiers, where the number of stacks has a slight effect on the value of $Act$ while the increase in the number of stack tiers brings about a rapid increase of the value of $Act$. The errors between the $SPFH$ algorithm and the $LL$ heuristic also have the same trend.

5.2.2. Effectiveness validation under large-batch instances

In this case, the configuration sizes of the experimental instances vary from $S=10,12$ stacks and $T=8,\dots ,10$ tiers. These experimental parameters meet the current production conditions of the busier container terminals and their development trends.

Table 6 summarises the validation of the algorithm $SPFH$ and the heuristic $LL$ for the large-size instances (each instance has $8,\dots ,10$ operation rounds and each round has $5,\dots ,12$ target containers) in the real-time operation scenarios. The column titled “MT” refers to the runtime of the most time-consuming instance out of 30 instances. The column “AT” refers to the average runtime of 30 instances. The meanings of other variables, namely I, $IB$, $I_{eb}$, and $Act$, are the same as those in the previous tables.

The results in Table 6 imply that both algorithms $SPFH$ and $LL$ can solve most instances quickly. However, when the number of target containers per round exceeds 9, the solution time of a few cases exceeds 3600 seconds, which means that the algorithms can no longer meet the needs of real-time optimization operations. Fortunately, in the actual operation scenario, there are rarely more than nine target containers during a real-time operation round.

The reason for the time-consuming solution process in the large-size batch instances is that the flexible pickup order mechanism requires iterating through all possible target pickup orders during the real-time operation round to explore the optimal operation plan. The number of iterations of the solution process increases rapidly with the target container number (n), which is approximately equal to $n!$. Two perspectives can be considered to solve this problem: First, reducing the solution quality, using heuristics for selecting target containers (e.g., always retrieving the target containers with the lowest number of blocking containers) instead of complete iteration to improve the solution efficiency and second, dividing the current larger batch into several smaller batches before processing, e.g., the current batch of 10 containers is divided into two small batches of 5 containers.

6. Conclusion

Considering the complexity of solving the CRP and improving the solution efficiency, the problems related to optimizing the real-time pick-up operations of inbound containers based on the arrival of ETs to reduce the number of relocations from a new perspective are addressed. Influenced by the uncertainty of the arrival time of ETs and the limited number of yard cranes configured at the terminal, the pick-up operation process of inbound containers in a bay is multi-round. For this reason, this study develops a dynamic programming model for the CRP and, on this basis, derives an upper bound for the optimal solution of the model, i.e., the actual number of relocations generated with each real-time operation round and the incremental number of new blocking containers caused by relocations. In the real-time operation scenario, where the ETs have arrived at the terminal, the model allows the adjustment of the pick-up order of target containers within the real-time operation round and selects the optimal pick-up order to obtain the least number of relocations as well as generates the least number of new blocking containers. Two algorithms ($LL$ and $SPFH$) and two relocation rules ($MSS$ and $FSS$) are proposed to solve the model, where the $LL$ heuristic is a basis solution method. The $SPFH$ algorithm is an augmented heuristic proposed by incorporating two relocation rules ($MSS$ and $FSS$) on top of the $LL$ heuristic. Numerical experiments show that all the proposed algorithms are effective in solving the problem to be solved, with the $SPFH$ algorithm showing superior performance in most cases. Optimizing only for the real-time operation round can reduce the size of the solving process and improve the efficiency of solving the model. Selecting the optimal pick-up order helps to reduce the total number of relocations. For large-scale instances, our methods are still feasible and practical.

Choosing a reasonable pick-up order can significantly reduce the number of relocations, but generating all possible orders for the target containers is time-consuming. The experimental results show that when the number of arrived ETs exceeds nine during an operation round, the solution time is too long in some instances, which will seriously affect real-time operation efficiency. The reason is that the target containers are scattered in each stack, which leads to the possible pick-up orders of the target containers ($n!$) being too large. There are two proposals for this problem: the first one has been mentioned in the previous section, which directly divides the target containers into two small batches according to the arrival time of the ETs; the second one is more flexible, i.e., if the ETs arrive before the yard crane starting operations, the corresponding target containers are divided into the first batch, and other ETs arrive after that, the related containers are divided into second batch, and so on.

Compared with the traditional CRP, which focuses on solving the pre-optimized operation plan for the overall retrieval process of a bay, real-time operations can achieve a better reduction of rehandles with the help of the arrival information of the ETs and the adjustable pick-up orders. Moreover, this paper only considers cases where all the containers in a bay have scheduled their retrieval time windows in advance. It is difficult for the terminal to direct the customers’ appointment time, and very often, some containers in a bay have been booked for pick-up time, and the other containers have yet to. Therefore, in this case of mixing the containers’ deterministic and uncertain pick-up information in a bay, the method to reduce the relocations will be an exciting project for future research.