Chapter 14. Approximation Algorithms

Classical sequential approximation algorithms: 2-approximation for vertex cover, greedy $H_n$ for set cover, FPTAS for knapsack.

This page: Classical forms. View LLP forms »

This page collects the classical sequential algorithms for three canonical NP-hard optimization problems: minimum Vertex Cover, minimum Set Cover, and $0/1$ Knapsack. The lattice-linear (LLP) reformulations live on the LLP companion page.

The approximation ratio

For a minimization problem, an algorithm $A$ is an $\alpha$-approximation ($\alpha \geq 1$) if $A(I) \leq \alpha \cdot OPT(I)$ on every instance $I$. For a maximization problem the condition is $A(I) \geq OPT(I) / \alpha$. A polynomial-time exact algorithm is a $1$-approximation; for an NP-hard problem we expect $\alpha > 1$. Some problems admit constant-factor approximations; some admit only $O(\log n)$; some admit no constant-factor approximation at all (assuming P $\neq$ NP).

ApproxVertexCover (2-approximation via maximal matching)

Given a graph $G = (V, E)$, repeatedly pick any uncovered edge, add both endpoints to the cover, and delete every incident edge. The set $F$ of edges picked forms a matching, $|C| = 2|F|$, and any vertex cover must contain at least one endpoint of every edge of $F$, so $OPT \geq |F|$. Therefore $|C| \leq 2 \cdot OPT$.

Time complexity: $O(|V| + |E|)$ with the right edge data structure.

boolean[] ApproxVertexCover(int[][] adj) {
  // Greedily pick endpoints of any uncovered edge.
  // Standard 2-approximation; see Section 14.2.
}

ApproxSetCover (greedy $H_n$-approximation)

Given a universe $U$ of $n$ elements and a family of $m$ subsets $S_1, \ldots, S_m \subseteq U$, repeatedly pick the set that covers the largest number of currently-uncovered elements; remove those elements and repeat. The cover returned has size at most $H_n \cdot OPT$, where $H_n = 1 + 1/2 + \cdots + 1/n \approx \ln n$. Feige (1998) proved this is essentially tight: no polynomial algorithm achieves $(1 - \epsilon)\ln n$ unless P $=$ NP.

Time complexity: $O(m \cdot n)$ per round; the number of rounds is at most $\min(m, n)$.

boolean[] ApproxSetCover(int[][] S) {
  // Greedy: at each step pick the set with maximum
  // uncovered-element count.  Returns selection vector G.
}

FPTAS-Knapsack (fully polynomial-time approximation scheme)

For any $\epsilon > 0$, scale item values to $v'_i = \lfloor v_i \cdot n / (\epsilon \cdot M) \rfloor$ where $M = \max_i v_i$, then run the standard exact $O(n \cdot \sum v'_i)$ DP on the scaled values and recover the chosen items by backtracking. The output is a feasible packing with value at least $(1 - \epsilon) \cdot OPT$. The total running time is polynomial in both $n$ and $1/\epsilon$ — the defining property of an FPTAS.

Time complexity: $O(n^3 / \epsilon)$.

boolean[] FPTASKnapsack(int[] w, int[] v, int W,
                        int epsNum, int epsDen) {
  // Scaled-value DP: v'[i] = (v[i] * n * epsDen) / (epsNum * M).
  // Solve exactly on v', backtrack to recover the selection.
}

Hardness of approximation

The constants above are essentially the best one can hope for assuming P $\neq$ NP. Håstad (2001) proved that Max-3-SAT cannot be approximated beyond $7/8 + \epsilon$ in polynomial time. Feige's $(1 - \alpha) \ln n$ lower bound for Set Cover already prohibits a constant-factor algorithm for that problem. Vertex Cover has resisted improvement below $2 - \epsilon$ since the 1970s, and the Unique Games Conjecture predicts that $2$ is in fact the right answer.