A Surprisal-Based Greedy Heuristic for the Set Covering Problem

1. Introduction

The Set Covering Problem (SCP) is a classical combinatorial optimization problem defined as follows. Let I be a set of m items and $J=\{S_1,S_2\dots ,S_n\}$ a collection of n subsets of I where each subset $S_j$ ($j=1,\cdots ,n$) is associated to a non-negative cost $c_j$. The SCP amounts to find a minimum cost subcollection of J that covers all the elements of I at minimum cost, the cost being defined as the sum of subsets cost.

The SCP finds applications in many fields. One of the most important is Crew Scheduling, where SCP provides a minumum-cost set of crews in order to cover a given set of trips. These problems include Airline Crew Scheduling (see, e.g., Rubin [1] and Marchiori [2]) and Railway Crew Scheduling (see, e.g., Caprara [3]). Other applications are the Winner Determination Problem in combinatorial auctions, a class of sales mechanisms (Abrache et al. [4]) and Vehicle Routing (Foster et al. [5], Cacchiani et al. [6] and Bai et al. [7]). The SCP is also relevant in a number of Production Planning Problems, as described by Vemuganti in [8], where it has often to be solved in real-time.

Garey and Johnson in [9] have proven that the SCP is NP-hard in the strong sense. Exact algorithms are mostly based on branch-and-bound and branch-and-cut techniques. Etcheberry [10] utilizes sub-gradient optimization in a branch-and-bound framework. Balas and Ho [11] present a procedure based on cutting planes from conditional bounds, i.e., valid lower bounds if the constraint set is amended by certain inequalities. Beasley [12] introduces an algorithm which blends dual ascent, subgradient optimization and linear programming. In [13], Beasley and Jornsten incorporate the [12] algorithm into a Lagrangian heuristic. Fisher and Kedia [14] use continuous heuristics applied to the dual of the linear programming relaxation, obtaining lower bounds for a branch and bound algorithm. Finally, we mention Balas and Carrera [15] with their procedure based on a dynamic subgradient optimization and branch and bound. These algorithms were tested on instances involving up to 200 rows and 2000 columns in the case of Balas and Fisher’s algorithms and 400 rows and 4000 columns in [12,13,15]. Among these algorithms the fastest one is the Balas and Carrera’s algorithm, with an average time in the order of 100 seconds on small instances and 1000 seconds on largest ones (on a Cray-1S computer). Caprara [16] compared these methods with the general-purpose ILP solvers CPLEX 4.0.8 and MINTO 2.3, observing that the latter ones have execution times competitive with that of the best exact algorithms for the SCP in literature.

In most industrial applications it is important to rely on heuristic methods, in order to obtain “good” solutions quickly enough to meet the expectations of decision-makers. To this purpose, many heuristics have been presented in the literature. The classical greedy algorithm proposed by Chvatal [17] sequentially inserts the set with a minimum score in the solution. Chvatal proved that the worst case performance ratio does not exceed $H\left(d\right)={\sum }_{i=1}^d\frac{1}{i}$, where d is the size of the largest set. More recently Kordalewski [18] described a new approximation heuristics for the SCP. His algorithm involves the same scheme of Chvatal’s procedure, but modifies the score by including a new parameter named difficulty. Wang et al. [19] presented the TS-IDS algorithm designed for deep web crawling and then Singhania [20] tested it in a resource management application. Feo and Resende [21] present a Greedy Randomized Adaptive Procedure (GRASP), in which they first construct an initial solution through an adaptive randomized greedy function and then apply a local search procedure. Haouari and Chaouachi [22] introduce PROGRES, a probabilistic greedy search heuristic which uses diversification schemes along with a learning strategy.

Regarding Lagrangian heuristics, we mention the algorithm developed by Beasley [23] and then improved by Haddadi [24], which consists of a subgradient optimization procedure coupled with a greedy algorithm and Lagrangian cost fixing. A similar procedure is designed by Caprara et al. [25], which includes three phases, subgradient, heuristic and column fixing, followed by a refining procedure. Beasley and Chu [26] propose a genetic algorithm in which a variable mutation rate and two new operators are defined. Similarly Aickelin [27] describes an indirect genetic algorithm. In this procedure actual solutions are found by an external decoder function and then an another indirect optimization layer is used to improve the result. Lastly, we mention Meta-Raps, introduced by Lan et al. [28], an iterative search procedure that uses randomness as a way to avoid local optima. All the mentioned heuristics present calculation times not compatible with real contexts. For example Caprara’s algorithm [25] produces solutions with an average computing time of about 400 seconds (on a DECstation 5000/240 CPU), if executed on non-unicost instances from Beasley’s OR Library, with $500\times 5000$ and $1000\times 10000$ as matrix sizes. Indeed, the difficulty of the problem leads to very high computational costs, which has led academics to research heuristics and meta-heuristics capable of obtaining good solutions, as close as possible to the optimal, in a very short time, in order to tackle real-time applications. To this respect, it is worth noting the paper by Grossman and Wool [29] in which a comparative study of eight approximation algorithms for the unicost SCP is proposed. Among these there were several greedy variants, fractional relaxations and randomized algorithms. Other investigations have been carried out over the years are: Galinier et al. [30], who study a variant of SCP, called the Large Set Covering Problem (LSCP), in which sets are possibly infinite; Lanza-Gutierrez et al. [31], which are interested in the difficulty of applying metaheuristics designed for solving continuous optimization problem to the SCP; Sundar et al. [32] who propose another algorithm to solve the SCP by combining an artificial bee colony (ABC) algorithm with local search.

In this paper, we exploit concepts from Information Theory (see Borda [33]) to improve the Chvatal’s greedy algorithm. Our purpose is to devise a heuristic able to improve the quality of the solution while retaining similar execution times to those of Chvatal’s algorithm, making it suitable for real-time applications. In particular, our algorithm, named Surprisal-Based Greedy Heuristic (SBH), introduces a surprisal measure, also known as self-information, to account partly for the problem structure while constructing the solution. We compare the proposed heuristic with three other greedy algorithms, namely the Chvatal’s greedy procedure [17], the Kordalewski’s algorithm [18] and the Altgreedy procedure [29] for unicost problems. We emphasize that SBH improves the classical Chvatal’s greedy algorithm [17] in terms of objective function and has the same scalability in computation time while the Kordalewski’s algorithm produces slightly better solutions but its computation times are much higher than those of the SBH algorithm, making it impractical for real-time applications.

The reminder of the article is organized as follows: in the Section 2 we describe the three algorithms involved in our analysis and illustrate SBH. Section 3 presents an experimental campaign which compares the greedy algorithms mentioned above. Finally, Section 4 reports some conclusions.

2. Surprisal-Based Greedy Heuristic

2.1. Problem Formulation

The SCP can be formulated as follows. In addition to the notation introduced in Section 1, let $a_{ij}$ be a constant equal to 1 if item i is covered by subset j and 0 otherwise. Moreover, let $x_j$ denote a binary variable defined as follows:

$$x_j=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\mathrm{column}\mathrm{j}\mathrm{is}\mathrm{selected},\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

An SCP formulation is:

$$\begin{array}{cc}\hfill \mathrm{minimize}\sum _{j\in J}& c_j{x}_j\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill \sum _{j\in J}{a}_{ij}{x}_j\ge 1& i\in I,\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill x_j\in \{0,1\}& j\in J,\hfill \end{array}$$

(3)

where (1) aims to minimize the total cost of the selected columns and (2) imposes that every row is covered by at least one column.

Algorithm 1 Chvatal’s greedy algorithm

2.2. Greedy Algorithms

As we explained in the previous section, we are interested in greedy procedures in order to produce good solutions in a very short time, suitable for real-time applications. SCP greedy algorithms are sequential procedure that identify the best unselected column w.r.t. to a given score and then insert it in the solution set.

Let $I_j$ be the set of rows covered by column j and $J_i$ the set of columns covering row i. Figure 1 shows the pseudocode of the Chvatal’s greedy algorithm [17]. Each column j is given a score equal to the column cost $c_j$ divided by the number of rows $I_j$ covered by j. At each step the algorithm inserts the column $j^{*}$ with the minimum score in the solution set.

A variant of the Chvatal’s procedure for unicost problems was suggested by Grossman and Wool [29] with the name of Altgreedy. This algorithm is composed by two main steps: in a first phase, the column with the highest number of covered rows is inserted in the solution; then some columns are removed from the solution set according to lexicographic order as long as the number of the new uncovered rows remains smaller than the last number of new rows covered.

More recently, Kordalewski [18] proposed a new greedy heuristic which is a recursive procedure that introduces two new terms: valuation and difficulty. In the first step, valuation is computed for all columns j by dividing the number of rows, covered by j, by the column cost, like in Chvatal’s score. For each row i is defined a parameter, difficulty, which is the inverse of the sum of the valuations of the sets covering i, used to indicate how difficult it might be to cover that row. This is based on the observation that a low valuation implies a low probability of selection. Valuation v can be computed as:

$$v_j={\displaystyle \frac{{\sum }_{i\in I_j}{d}_i}{c_j}}$$

while difficulty d will be only updated with the new valuations.

2.3. The SBH Algorithm

In this section, we describe the Surprisal-Based Greedy Heuristic SBH, that constitutes an improvement on the classic Chvatal’s greedy procedure. As illustrated in Section 2.2, Chvatal’s algorithm assigns each column j a score equal to the unit cost to cover the rows in $I_j$. Then it inserts iteratively the columns with the lowest score in the solution set. However, this approach is flawed when rows in $I_j$ are poorly covered. Indeed it does not consider the probability that rows $i\in I_j$ are covered by other columns $j^{\prime }\in J_i$. Our algorithm aims to correct this by introducing an additional term expressing the “surprisal” that a column j is selected. Therefore our score considers two aspects: the cost of a column j and the probability that the rows in $I_j$ can be covered by other columns.

To describe formally our procedure, we introduce some concepts from Information Theory. The term information refers to any message which gives details in an uncertain problem and is closely related with the probability of occurrence of an uncertain event. Information is an additive and non-negative measure which is equal to 0 when the event is certain and it grows when its probability decreases. More specifically, given an event A with probability to occur ${\mathit{p}}_A$, the self-information ${\mathcal{I}}_A$ is defined as:

$${\mathcal{I}}_A=-log\left({\mathit{p}}_A\right).$$

(4)

Self-information is also called surprisal because it expresses the “surprise” of seeing event A as the outcome of an experiment. In the SBH algorithm, at each stage we compute the surprisal of each column. The columns containing row i are considered independent of each other, so the probability of selecting one of them (denoted as event $\overline{A}$) is

$${\mathit{p}}_{\overline{A}}={\displaystyle \frac{1}{|J_i|}}.$$

(5)

Therefore the opposite event, i.e., select row i with a column different from the current one, is:

$${\mathit{p}}_A=1-{\displaystyle \frac{1}{|J_i|}}={\displaystyle \frac{|J_i|-1}{|J_i|}}.$$

(6)

The self-information measure contained in this event is:

$${\mathcal{I}}_i=-log\left(\frac{|J_i|-1}{|J_i|}\right).$$

(7)

Thanks to the additivity of the self-information measure, surprisal of a column j can be written as:

$${\mathcal{I}}_j=\sum _{i\in I_j}{\mathcal{I}}_i=\sum _{i\in I_j}-log\left(\frac{|J_i|-1}{|J_i|}\right).$$

(8)

We modify Chvatal’s cost of column j, i.e., ${\displaystyle \frac{c_j}{|I_j|}}$, by introducing the surprisal of j to the denominator, in order to favour columns with a high self-information. In particular, at each step we select the column that minimizes:

$$\underset{j\in J}{min}\frac{c_j}{|I_j|·{\mathcal{I}}_j},$$

(9)

which is equivalent to

$$\underset{j\in J}{min}\frac{c_j}{|I_j|}\prod _{i\in I_j}\frac{|J_i|-1}{|J_i|}.$$

(10)

This formulation is the same of minimizing the probability of the intersection of independent events, each of which is to select a column, other than the current one, covering row i. Two extreme cases can occur:

• if column j is the only one covering a row $i\in I_j$, there is no surprise that it will be selected: in this case ${\mathcal{I}}_j$ will be high and the modified cost (9) of column j will be 0 so that column j will be included in the solution;
• if, on the other hand, all rows $i\in I_j$ are covered by a high number of other columns $j^{\prime }\in J_i$, surprisal ${\mathcal{I}}_j$ will be very low. In this case, the cost attributed to column j will be greater than its Chvatal’s cost.

To illustrate this concept, we now present a numerical example. Let

$$\left(a_{ij}\right)=\left[\begin{array}{cccc}1& 1& 0& 0\\ 0& 0& 0& 1\\ 1& 1& 1& 0\\ 1& 0& 1& 1\end{array}\right],\left(c_j\right)=\left[3125\right]$$

be the coverage matrix and the column cost vector. We denote by $C{H}_{score}^i$ and $SB{H}_{score}^i$ Chvatal and SBH score vectors, respectively, at the i-th iteration, where a hyphen is inserted to indicate that the corresponding column has already been selected. The three iterations of the Chvatal’s algorithm produce:

$$\begin{array}{cc}\hfill & C{H}_{score}^1=\left[1;\frac{1}{2};1;\frac{5}{2}\right]\hfill \\ \hfill & C{H}_{score}^2=\left[3;-;2;\frac{5}{2}\right]\hfill \\ \hfill & C{H}_{score}^3=\left[-;-;-;5\right]\hfill \end{array}$$

and columns $2,3,4$ are selected with a total cost equal to 8. On the other hand, our SBH algorithm gives:

$$\begin{array}{cc}\hfill & SB{H}_{score}^1=\left[\frac{2}{9};\frac{1}{6};\frac{4}{9};0\right]\hfill \\ \hfill & SB{H}_{score}^2=\left[\frac{1}{2};\frac{1}{6};\frac{4}{3};-\right]\hfill \end{array}$$

Hence, our algorithm selects only two columns (4 and 2), with a total cost of 6. Therefore, SBH outperforms Chvatal’s procedure because the latter cannot recognize the column that must necessarily be part of the solution.

3. Experimental Results

The aim of our computational experiments has been to assess the performance of the SBH heuristic procedure with respect to the other greedy heuristics proposed in literature. We implemented the heuristics in C++ and performed our experiments on a stand-alone Linux machine with 4 cores processor clocked at 3 GHz and equipped with 16 GB of RAM. The algorithm has been tested on 77 instances from Beasley’s OR Library [34]. Table 1 describes the main features of the test instances and, in particular, the column density, i.e., the percentage of ones in matrix a and column range, i.e., the minimum and maximum value of objective function coefficients. The remaining column headings are self-explanatory. Instances are divided into sets having sizes ranging from $200\times 1000$ to $1000\times 10000$. Set E contains small unicost instances of size $50\times 500$. Sets 4, 5 and 6 were generated by Balas and Ho [11] and consist of small instances with low density while sets A to E come from Beasley [12]. The remaining instances (sets $NRE$ to $NRH$) are from [23]. Such instances are significantly larger and optimal solutions are not available. Similarly, Table 2 reports features of seven large scale real-word instances derived from crew-scheduling problem [25].

We compare SBH with the Chvatal’s procedure [17] (CH) and the heuristic by Kordalewski [18] (KORD). Table 3, Table 4 and Table 5 report the computational results for each instance under the following headings:

• Instance: the name of the instance where the string before “dot” refers to the set which the instance belongs to;
• BS: objective function value of the best known solution;
• SOL: the objective function value of the best solution determined by the heuristic;
• TIME: the execution time in seconds;
• GAP: percentage gap between BS and the SOL value, i.e.,

  $$GAP=100\times \frac{SOL-BS}{BS}$$

Columns “SBH vs CH” and “SBH vs KORD” report the percentage improvement of SBH w.r.t. CH and KORD, respectively. Regarding Table 3 it is worth noting that our heuristic, compared to Chvatal’s greedy procedure, has a smaller gap ranging from $12.65\%$ to $11.03\%$, with an average improvement of $1.42\%$. Among these instances, SBH provides a better solution than [17] in 19 out of 24 instances problems. We point out that the best objective function value is given by the Kordalewski’s algorithm which is slightly better than our SBH procedure (by only $0.59\%$), but is slower.

Similar observations can be done for Table 4. Here, SBH performs better even though it differs from the Kordalewski algorithm only by $0.07\%$. Comparing SBH with CH, it is worth noting that only in 4 instances out of 45 SBH obtained a worse solution. SBH comes close to the optimal solution with an average gap of $10.69\%$ and is better than CH by $2.62\%$. Execution time of all these instances averages $0.113$ seconds for CH, $0.230$ seconds for the Kordalewski procedure and $0.564$ seconds for SBH. Increasing the size of the instances (which is the case of the rail problems), the Kordalewski’s algorithm becomes much slower. Consequently, on these instances we compare only the CH and SBH heuristics. On these instances, our SBH algorithm provides an average objective function improvement of $5.82\%$ with comparable execution times.

We now compare the algorithms on unicost instances, obtained by setting the cost of all columns equal to 1, like in Grossman and Wool’s paper [29]. Results are showed in Tables Table 6, Table 7 and Table 8, where the subdivision of instances is the same as before. The additional column “SBH vs ALTG” reports the percentage improvement of SBH w.r.t. ALTG algorithm. Looking at Table 6, it worth noting that the heuristic which performs better is that of Kordalewski. Indeed, our heuristic SBH is worst than KORD of about $3.49\%$ while it is better then the other two greedy procedures with a gap of $1.15\%$. Here, computation times are all comparable and range between $0.002$ and $0.007$ seconds. SBH improves its performance on larger instances, as shown in Table 7 and Table 8. In particular, it yields an average improvement of $1.50\%$ on CH and ALTG ([29]) on $scp$ instances, and respectively $1.39\%$ and $12.97\%$ on $rail$ instances. Comparing SBH and $KORD$ on the $scp$ instances, we observe that they are very similar with a $0.07\%$ improvement. On the largest instances (Table 8) it emerges that the computational time of KORD makes it impractical for real-time applications.

4. Conclusions

In this paper, we have proposed a new greedy heuristic, SBH, an improvement of the classical greedy algorithm proposed by Chvatal [17]. We showed that in the vast majority of the test instances SBH generates better solutions then other greedy algorithms such as the Kordalewski’s algorithm [18] and Altgreedy [29]. Computational tests have also shown that the Kordalewski’s algorithm is not suitable for real-time application since it presents very large execution times while our SBH algorithm runs in few seconds even on very large instances.