未知算法设计科dye

1、1Linear Programming & FlowsInstructor: yedeshi 2Linear ProgrammingWhat is it? Quintessential tool for optimal allocation of scarce resources, among a number of competing activities. (e.g., see ORF 307) Powerful and general problem-solving method.shortest path, max flow, min cost flow, generalized fl

2、ow, modity flow, MST, matching, 2-person zero sum gamesWhy significant? Fast commercial solvers: CPLEX, OSL, Lindo. Powerful modeling languages: AMPL, GAMS. Ranked among most important scientific advances of 20th century. Also a general tool for attacking NP-hard optimization problems. Dominates wor

3、ld of industry.ex: Delta claims saving $100 million per year using LP3Example4Geometry Observation. Regardless of objective function (convex) coefficients, an optimal solution occurs at an extreme point.5Standard and slack forms Standard formIn standard form, we are given n real numbers c1, c2, ., c

4、n; m real numbers b1, b2, ., bm; and mn real numbers aij for i = 1, 2, ., m and j = 1, 2, ., n. We wish to find n real numbers x1, x2, ., xn that6Standard and slack forms Standard formIn standard form, we are given n real numbers c1, c2, ., cn; m real numbers b1, b2, ., bm; and mn real numbers aij f

5、or i = 1, 2, ., m and j = 1, 2, ., n. We wish to find n real numbers x1, x2, ., xn that7slack form In standard form,Slack variable s, such that 8The simplex algorithm Simplex algorithm. (George Dantzig, 1947) Developed shortly after World War II in response to logistical problems. Used for 1948 Berl

6、in airlift.Generic algorithm. Start at some extreme point. Pivot from one extreme point to a neighboring one. Repeat until optimal.How to implement? Linear algebra.never decrease objective func9Illustration of SimplexStep1: Convert the LP problem to a system of linear equations (slack form). Step 2:

7、 Set up the initial tableau. Step 3: Find the PIVOT Step 4: Form RATIOS (quotients) for each row: divide the right-most number by the number in the pivot column of that row. Step 5: The PIVOT ROW is the row with the smallest NON-NEGATIVE ratio (quotient).Step 6: If all indicators (in the bottom row)

8、 are non-negative, STOP. Step 7: Otherwise, goes to Step 3.10Step 1Original problem:Slack form:11Step 221 11001442 30102825 500130-1-2+10000SIMPLEX TABLEAU.Compare RED symbolswith Z = x1 + 2x2 - x3. 12Step 3: Pivot21 11001442 30102825 500130-1-2+10000Pivot columnIndicator row13Step 3: Pivot ratios21

9、 11001442 30102825 500130-1-2+10000Pivot columnIndicator rowRatios:141 =14282 =14305 =6Pivot14Step 4: entering variable21 11001442 30102825 500130-1-2+10000r1:r2:r3:r4:21 11001442 3010282/51 1001/56-1-2+10000R3=(r3 /5)15Step 4: leaving variabler1:r2:R3:r4:21 11001442 3010282/51 1001/56-1-2+10000R1=r

10、1 - R3R2=r2-2R3R4=r4+2R38/50 010-1/5816/50 101-2/5162/51 1001/56-1/503002/512R1:R2:R3:R4:16Step 3 again8/50 010-1/5816/50 101-2/5162/51 1001/56-1/503002/512R1:R2:R3:R4:Pivot columnIndicator row17Step 3 again8/50 010-1/5816/50 101-2/5162/51 1001/56-1/503002/512R1:R2:R3:R4:Pivot columnIndicator rowRat

11、ios:8(8/5) =516(16/5) =56 (2/5) =15Tie: choose arbitrary18Step 3 again8/50 010-1/5816/50 101-2/5162/51 1001/56-1/503002/512R1:R2:R3:R4:8/50 010-1/5810 5/1605/16-1/852/51 1001/56-1/503002/512R1:R2:R3:R4:R2=(R2 /(16/5)19Step 3 again8/50 010-1/5810 5/1605/16-1/852/51 1001/56-1/503002/512R1:R2:R3:R4:00

12、-1/21-1/20010 5/1605/16-1/8501 7/80-1/81/440049/1601/163/813R1:R2:R3:R4: R1=R1 (8/5)R2R3=R3-(2/5)R2 R4=R4+(1/2)R220STOP00 -1/21-1/20010 5/1605/16-1/8501 7/80-1/81/440049/1601/163/813R1:R2:R3:R4:x1x2x3s1s2s3x3=s2=s3=0 x1=5, x2=4, s1=0 z =13Finally, the optimal solution of LP is 21History of LP1939. P

13、roduction, planning. (Kantorovich, USSR)Propaganda to make paper more palatable to communist censors.Kantorovich awarded 1975 Nobel prize in Economics forcontributions to the theory of optimum allocation of resources. Staple in MBA curriculum. Used by most large companies and other profit maximizers

14、.22History1939. Production, planning. (Kantorovich)1947. Simplex algorithm. (Dantzig)1950. Applications in many fields. Military logistics. Operations research. Control theory. Filter design. Structural optimization.23History1939. Production, planning. (Kantorovich)1947. Simplex algorithm. (Dantzig)

15、1950. Applications in many fields.1979. Ellipsoid algorithm. (Khachian) Geometric divide-and-conquer. Solvable in polynomial time: O(n4 L) bit operations. n = # variables L = # bits in input Theoretical tour de force, not remotely practical.24History1939. Production, planning. (Kantorovich)1947. Sim

16、plex algorithm. (Dantzig)1950. Applications in many fields.1979. Ellipsoid algorithm. (Khachian)1984. Projective scaling algorithm. (Karmarkar) O(n3.5 L). Efficient implementations possible.25History1939. Production, planning. (Kantorovich)1947. Simplex algorithm. (Dantzig)1950. Applications in many

17、 fields.1979. Ellipsoid algorithm. (Khachian)1984. Projective scaling algorithm. (Karmarkar)1990. Interior point methods. O(n3 L) and practical. Extends to even more general problems.26PerspectiveLP is near the deep waters of pleteness. Solvable in polynomial time. Known for less than 28 years.Integ

18、er linear programming. LP with integrality requirement. NP-hard.27Formulating problems as linear programs Maximum flow:we are given a directed graph G = (V, E) in which each edge (u, v) E has a nonnegative capacity c(u, v) 0, and two distinguished vertices, a sink s and a source t. The goal is to fi

19、nd s-t flow of maximum value.28 China Rail Network29Maximum Flow and Minimum CutMax flow and min cut.Two very rich algorithmic problems.Cornerstone problems in combinatorial optimization.Beautiful mathematical duality. Minimum Cut ProblemFlow network.Abstraction for material flowing through the edge

20、s.G = (V, E) = directed graph, no parallel edges.Two distinguished nodes: s = source, t = sink.c(e) = capacity of edge e.30s234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 4capacitysourcesinkCutsDef. An s-t cut is a partition (A, B) of V with s A and t B.Def. The capacity of a cut (A, B) is:31s234567t

21、15 5 30 15 10 8 15 9 6 10 10 10 15 4 4 Capacity = 10 + 5 + 15 = 30 ACutsDef. An s-t cut is a partition (A, B) of V with s A and t B.Def. The capacity of a cut (A, B) is:32s234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 4 A Capacity = 9 + 15 + 8 + 30 = 62Minimum Cut ProblemMin s-t cut problem. Find an

22、s-t cut of minimum capacity.33s234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 4 A Capacity = 10 + 8 + 10 = 28FlowsDef. An s-t flow is a function that satisfies:For each e E: (capacity)For each v V s, t: (conservation)Def. The value of a flow f is: 34400000044000Value = 40capacityflows234567t 15 5 30 1

23、5 10 8 15 9 6 10 10 10 15 4 404FlowsDef. An s-t flow is a function that satisfies:For each e E: (capacity)For each v V s, t: (conservation)Def. The value of a flow f is: 3510661111038800011capacityflows234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 40Value = 244Maximum Flow ProblemMax flow problem. Fi

24、nd s-t flow of maximum value.36109914410489100014capacityflows234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 40Value = 28Flows and CutsFlow value lemma. Let f be any flow, and let (A, B) be any s-t cut. Then, the net flow sent across the cut is equal to the amount leaving s.3710661111038800011s234567t

25、 15 5 30 15 10 8 15 9 6 10 10 10 15 4 40Value = 244AFlows and CutsFlow value lemma. Let f be any flow, and let (A, B) be any s-t cut. Then, the net flow sent across the cut is equal to the amount leaving s.38106611038800011s234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 40 Value = 6 + 0 + 8 - 1 + 11 =

26、 24411AFlows and CutsFlow value lemma. Let f be any flow, and let (A, B) be any s-t cut. Then, the net flow sent across the cut is equal to the amount leaving s.3910661111038800011s234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 40 Value = 10 - 4 + 8 - 0 + 10 = 244AFlows and CutsFlow value lemma. Let f

27、 be any flow, and let (A, B) be any s-t cut. ThenPf. 40by flow conservation, all termsexcept v = s are 0Flows and CutsWeak duality. Let f be any flow, and let (A, B) be any s-t cut. Then the value of the flow is at most the capacity of the cut.41Cut capacity = 30 Flow value 30 s234567t 15 5 30 15 10

28、 8 15 9 6 10 10 10 15 4 4Capacity = 30AFlows and CutsWeak duality. Let f be any flow. Then, for any s-t cut (A, B) we havev(f) cap(A, B).Pf.42stAB 76 84Certificate of OptimalityCorollary. Let f be any flow, and let (A, B) be any cut.If v(f) = cap(A, B), then f is a max flow and (A, B) is a min cut.4

29、3Value of flow = 28Cut capacity = 28 Flow value 28109914410489100014s234567t 15 5 30 15 10 8 15 9 6 10 10 10 15 4 40A Bipartite MatchingMatchingMatching.Input: undirected graph G = (V, E).M E is a matching if each node appears in at most one edge in M.Max matching: find a max cardinality matching.45

30、Bipartite MatchingBipartite matching.Input: undirected, bipartite graph G = (L R, E).M E is a matching if each node appears in at most one edge in M.Max matching: find a max cardinality matching.461351352424matching1-2, 3-1, 4-5 RLBipartite MatchingBipartite matching.Input: undirected, bipartite gra

31、ph G = (L R, E).M E is a matching if each node appears in at most edge in M.Max matching: find a max cardinality matching.471351352424RLmax matching1-1, 2-2, 3-3 4-4 Bipartite MatchingMax flow formulation.Create digraph G = (L R s, t, E ).Direct all edges from L to R, and assign infinite (or unit) c

32、apacity.Add source s, and unit capacity edges from s to each node in L.Add sink t, and unit capacity edges from each node in R to t.48s135135t242411RLGBipartite Matching: Proof of CorrectnessTheorem. Max cardinality matching in G = value of max flow in G.Pf. Given max matching M of cardinality k.Con

33、sider flow f that sends 1 unit along each of k paths.f is a flow, and has cardinality k. 49s135135t2424111351352424GGBipartite Matching: Proof of CorrectnessTheorem. Max cardinality matching in G = value of max flow in G.Pf. Let f be a max flow in G of value k.Integrality theorem k is integral and c

34、an assume f is 0-1.Consider M = set of edges from L to R with f(e) = 1.each node in L and R participates in at most one edge in M|M| = k: consider cut (L s, R t) 501351352424Gs135135t242411G Disjoint PathsEdge Disjoint PathsDisjoint path problem. Given a digraph G = (V, E) and two nodes s and t, fin

35、d the max number of edge-disjoint s-t paths.Def. Two paths are edge-disjoint if they have no edge in common.Ex: communication networks.52s234567tEdge Disjoint PathsDisjoint path problem. Given a digraph G = (V, E) and two nodes s and t, find the max number of edge-disjoint s-t paths.Def. Two paths a

36、re edge-disjoint if they have no edge in common.Ex: communication networks.53s234567tEdge Disjoint PathsMax flow formulation: assign unit capacity to every edge.Theorem. Max number edge-disjoint s-t paths equals max flow value.Pf. Suppose there are k edge-disjoint paths P1, . . . , Pk.Set f(e) = 1 i

37、f e participates in some path Pi ; else set f(e) = 0.Since paths are edge-disjoint, f is a flow of value k. 54st11111111111111Edge Disjoint PathsMax flow formulation: assign unit capacity to every edge.Theorem. Max number edge-disjoint s-t paths equals max flow value.Pf. Suppose max flow value is k.

38、Integrality theorem there exists 0-1 flow f of value k.Consider edge (s, u) with f(s, u) = 1.by conservation, there exists an edge (u, v) with f(u, v) = 1continue until reach t, always choosing a new edgeProduces k (not necessarily simple) edge-disjoint paths. 55st11111111111111can eliminate cycles

39、to get simple paths if desiredNetwork ConnectivityNetwork connectivity. Given a digraph G = (V, E) and two nodes s and t, find min number of edges whose removal disconnects t from s.Def. A set of edges F E disconnects t from s if all s-t paths uses at least on edge in F.56s234567tEdge Disjoint Paths

40、 and Network ConnectivityTheorem. Menger 1927 The max number of edge-disjoint s-t paths is equal to the min number of edges whose removal disconnects t from s.Pf. Suppose the removal of F E disconnects t from s, and |F| = k.All s-t paths use at least one edge of F. Hence, the number of edge-disjoint

41、 paths is at most k. 57s234567ts234567tDisjoint Paths and Network ConnectivityTheorem. Menger 1927 The max number of edge-disjoint s-t paths is equal to the min number of edges whose removal disconnects t from s.Pf. Suppose max number of edge-disjoint paths is k.Then max flow value is k.Max-flow min

42、-cut cut (A, B) of capacity k.Let F be set of edges going from A to B.|F| = k and disconnects t from s. 58s234567ts234567tA Extensions to Max FlowCirculation with DemandsCirculation with demands.Directed graph G = (V, E).Edge capacities c(e), e E.Node supply and demands d(v), v V.Def. A circulation

43、is a function that satisfies:For each e E:0 f(e) c(e) (capacity)For each v V: (conservation)Circulation problem: given (V, E, c, d), does there exist a circulation?60demand if d(v) 0; supply if d(v) 0; transshipment if d(v) = 0Circulation with DemandsNecessary condition: sum of supplies = sum of dem

44、ands.Pf. Sum conservation constraints for every demand node v.613106-7-811-64973100744667142flowcapacitydemandsupplyCirculation with DemandsMax flow formulation.62G:supply3106-7-811-6491007474demandCirculation with DemandsMax flow formulation.Add new source s and sink t.For each v with d(v) 0, add e

45、dge (v, t) with capacity d(v).Claim: G has circulation iff G has max flow of value D.63G:supply3106907474st1011786saturates all edgesleaving s and entering tdemandCirculation with DemandsIntegrality theorem. If all capacities and demands are integers, and there exists a circulation, then there exists one that is integer-valued.Pf. Follows from max flow formulation and int