DYNAMIC PROGRAMMING – ITS
PRINCIPLES, APPLICATIONS,
STRENGTHS, AND LIMITATIONS
BISWAJIT BHOWMIK
MIEEE, MIACSIT, MIAENG, MPASS, MIAOE, MIAS Sr. Lecturer
Department of Computer Science & Engineering Bengal College of Engineering and Technology
Durgapur – 713 212 India
Abstract
The massive increase in computation power over the last few decades has substantially enhanced our ability to solve complex problems with their performance evaluations in diverse areas of science and engineering. With the recent developments in the field of optimizations, these methods are now become lucrative to make decisions. Dynamic Programming is one of the elegant algorithm design standards and is powerful tool which yields classic algorithms for a variety of combinatorial optimization problems. In this paper fundamental working principles, major area of applications of this approach has been introduced. The strengths which make it more prevailing than the others is also opened up. Focusing the imperative drawbacks afterward comparison study of this algorithm design technique in this paper brings a general awareness to the implementation strategies.
Keywords: Principle of Optimality; Polynomial Break up; Subproblem; programming; Divide and Conquer; NP-hard.
1. INTRODUCTION
Those who cannot remember the past are doomed to repeat it.
--- George Santayana
Some well-devised algorithm design standards - such as Divide and Conquer, Greedy Methods, Graph Exploration, etc, yield classic algorithms for a range of optimization problems. However, a major drawback of these tools is that they can only be applied to very specific kinds of problems [6]. E.g. certain problems have recursive solutions that are very natural but simultaneously are inefficient too. This inefficiency is due to many identical recursive calls made during any given computation. This property, termed overlapping subproblems, is characteristic of many important problems [5, 14].
Therefore, we must turn to the sledgehammer of the algorithms craft – dynamic programming. It is the most powerful scheme and is an alternative way of looking at these discrete optimization problems that could be solved and is invoked in very broad applicability when more specialized methods fail. Predictably, this generality often moves towards with an optimal cost [6]. Problems having the overlapping subproblems property are almost always solved using dynamic programming [1].
We begin by providing a general insight into working principle of the dynamic programming approach. We then give a formal characterization of applications, strengths, and limitations.
2. WORKING PRINCIPLES
Dynamic programming was the brainchild of an American Mathematician, Richard Bellman, who described the way of solving problems where you need to find the best decisions one after another. The word Programming
Vol. 2(9), 2010, 4822-4826
i. Divide into Subproblems – The main problem is divided into a number of smaller, similar subproblems.
The solution to main problem is expressed in terms of the solution for the smaller subproblems. Stage wise solutions start with the smallest subproblems [3, 4].
ii. Construction of Table for Storage - The underlying idea of dynamic programming is to avoid
calculating the same stuff twice and usually a table of known results of subproblems is constructed for the purpose. Dynamic programming thus takes advantage of the duplication and arranges to solve each subproblem only once, saving the solution in table for later use [4, 25]. The key to competence of a dynamic programming algorithm is that once it computes the solution to a constrained version of the problem, it stores that solution in a table until the solution is no longer needed by any future computation. The initial solution is trivial [16]. This tells us that we trade space for time to avoid repeating the computation of a subproblem.
iii. Combining using Bottom-up means - Combining solutions of smallest subproblems obtain the solutions
to subproblems of increasing size. The process is continued until we arrive at the solution of the original problem [2, 4].
Dynamic programming involves selection of optimal decision rules that optimizes a certain performance criterion [4, 7]:
i. The Principle of Optimality – An optimal sequence of decisions is obtained iff each subsequence must
be optimal. That means if the initial state and decisions are optimal then the remaining decisions must constitute an optimal sequence w.r.t the state resulting from the first decision. Combinatorial problems may have this property but may exploit too much memory and/or time towards efficiency [15].
ii. Polynomial Break up - The original problem is divided into several subproblems. The division is done
in such a way that the total number of subproblems to be solved should be a polynomial or almost a polynomial number. This is done for efficient performance of dynamic programming.
Using the top-down view of dynamic programming, the first property mentioned above corresponds to be able to write down a recursive procedure for the problem that we want to solve. The second property makes clear in our mind that this recursive procedure builds only a polynomial number of different recursive calls [18].
3. APPLICATIONS
The versatility of the dynamic programming method is really appreciated by exposure to a wide variety of applications. My intent is to understand and contribute to the cutting edge research on optimization problems such as the classical optimal stopping problem and optimal resource allocation within the specific framework of approximate dynamic programming, complex decision making problems under uncertainty like reliability analysis, resource allocation, biological sequence manipulation and risk management [7, 17, 28].
Within this framework a variety of optimization techniques can be employed to solve particular aspects of a more general formulation in a multiplicity of areas [17, 18, 23, 26] such as –
(i) Information theory. (ii) Control theory. (iii) Bioinformatics. (iv) Operations research.
(v) Computer science - theory, graphics, Artificial Intelligence, etc.
More than the optimization problems described previously, the approach offers a general framework for analyzing many problems [1, 2, 6, 9, 11, 4, 5, 14, 20, 26, 29, 30] like -
(v)
Global Sequence Alignment.(vi)
Scoring Alignments.(vii)
Segmented Least Squares Problem.(viii)
Chain Matrix Multiplication.(ix)
Subset Sum.(x)
Independent sets in trees.(xi)
Shuffled Strings.(xii)
Optimal Capacity Expansion.(xiii)
Continuous State-Space Problems.(xiv)
Discounting Future Returns.(xv)
Edit Distance.(xvi)
Coin Change.(xvii)
Family Graph.(xviii)
0/1 Knapsack Problem.(xix)
Optimal Stopping Problems.(xx)
Optimal Binary Search Tree.(xxi)
Flow Shop Scheduling.(xxii)
Minimum Weighted Triangulation.(xxiii)
The Manhattan Tourist Problem.(xxiv)
Travelling Salesman Problem.(xxv)
Shortest Path Problems(a) Single Source Shortest Path. (b) Single Destination Shortest Path. (c) Single Pair Shortest Path. (d) All Pairs Shortest Path.
4. STRENGTHS
Creativity is necessary before we can distinguish that a particular problem can be casted effectively as a dynamic program. Even clever insights to restructure the formulation often are essential in useful solution [24, 25]. This idea of reusing subproblems is the main advantage of the dynamic programming paradigm over recursion. The simplicity what makes dynamic programming more appealing is both a full problem solving method and a subroutine solver in more complicated algorithmic solutions [19, 22].
The key to competence of the dynamic programming approach lies in a table that stores partial solutions for future references. Attractiveness of dynamic programming during the search for a solution on the other hand lays avoidance of full enumeration by clipping early partial decision solutions that cannot possibly lead to optimal solution. In a single word it makes the optimization procedure multistage in nature [13, 17, 22].
The most charisma involves selection of optimal decision rules: The Principle of Optimality and Polynomial Break up, which optimizes performance criterion. The approach is both a full problem solving method and a
subroutine solves [4, 14, 18]. These simplicities make dynamic programming technique more appealing in complicated algorithmic solutions that also we think about.
Dynamic programming is so powerful device that encourages tremendous growth in researches for solving sequential decision problems, and research related to dynamic programming has lead to fundamental advances in theory, numerical methods, and econometrics.
Vol. 2(9), 2010, 4822-4826
Finally though it is tedious to accomplish by hand, but dynamic programming is actually relatively efficient compared to a brute force listing of all possible combinations to find the best one [23].
5. LIMITATIONS
What kinds of problems can be solved using Dynamic Programming? Evidently, the answer is optimization problems. But the optimal solution involves solving a subproblem, and then it uses the optimal solution to that subproblem [18]. This key property of the solutions produced by dynamic programming is that they are time consistent. This is essentially due to direct implication of the principle of optimality [10]. Another drawback of this practice is that it works best on objects which are linearly ordered and cannot be rearranged such as characters in a string, points around the boundary of a polygon, matrices in a chain, the left-to-right order of leaves in a search tree, etc [15, 19]. The major shortcoming of making use of dynamic programming as a means is that it is often nontrivial to write code that evaluates the subproblems in the most efficient order [5, 25]. The challenge of devising a good solution method is in steps forward to make decisions what are the subproblems, how they would be computed and in what order. Apart from the obvious requirements - The Principle of Optimality and Polynomial Break up, an efficient dynamic programming induces only a “small” number of
distinct subproblems [4, 19].
6. COMPARISON STUDY
Undoubtedly it is generally speaking that DP approach is a most powerful optimization paradigm over the others. But what makes it so popular? The answer simply lays on its comparative study with other two popular techniques Divide-and-Conquer and Greedy Method [2, 4]. Like divide-and-conquer, dynamic programming results optimal solutions by combining the partial best possible solutions to sub-problems. Unlike the case in divide-and-conquer algorithms, immediate implementation of the recurrence results in identical recursive calls that are executed more than once [3]. The structure of dynamic programming is similar to divide-and-conquer, except that the subproblems to be solved are overlapping in nature which makes as a consequence different recursive paths to the same subproblems [11]. Thus, for solving a problem, divide-and-conquers are Independent problems, solve problems independently and recursively. Conversely, in dynamic programming sub-problems are dependent.
Greedy method is also a powerful technique for optimizations but not much like dynamic programming approach. In greedy, we solve a problem making greedy choices. After the choice is made the subproblem is arising. These choices may depend on previous choices. However, the choice is independent of the solutions to subproblems [1, 27, 16]. Top-down convention is normally used towards the feasible solution decreasing current problem size. Unlike greedy, choice is made at each step and bottom up approach is employed increasing problem size from smaller to larger subproblems answering optimal solutions. It is more powerful than greedy as it could be applicable to wide range of applications [4, 23, 28].
ACKNOWLEDGEMENTS
I would like to convey my heartily gratitude to Parag Kumar Guha Thakurta, Asst. Professor, Bibhas Sen, Asst. Professor, National Institute of Technology Durgapur, Niladri Sett, Research Scholar, Indian Institute of Technology Guwahati, Soumen Bag, Research Scholar, Indian Institute of Technology Kharagpur, and Dr. Subrata Trivedi, Asst. Professor of our institute for their valuable suggestions, remarks, criticisms, and comments towards improvement of quality of this paper.
REFERENCES
[1] Thomas H. Coremen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein, “Introduction to Algorithms”, PHI, 2nd Edn, 2008.
[2] Ellis Horowitz, Sartaj Sahani, Sanguthevar Rajasekaran, “Computer Algorithms”, University Press, 2nd Edn, 2008.
[3] M.H. Alsuwaiyel , “Algorithms Design Techniques and Analysis”, PHEI, 2002.
[4] B. Bhowmik, “Design and Analysis of Algorithms – WBUT Series”, 1st Edn, 2010 [in press].
[5] David B. Wagner, “Dynamic Programming”, THE MATHEMATICA JOURNAL, Miller Freeman Publications, 1995.
[6] S. Dasgupta, C.H. Papadimitriou, and U.V. Vazirani, “Algorithms”, TMGH, 5th reprint, 2009.
[9] Bertsekas, Dimitri, 1976, Dynamic Programming and Stochastic Control, New York: Academic Press.
[10] John Rust, Dynamic Programming, University of Maryland April 5, 2006.
[11] Chow, C.S. and Tsitsiklis, J.N. (1989) .The Complexity of Dynamic Programming. Journal of Complexity 5 466.488.
[12] Bertsekas, D. P. (1995) Dynamic Programming and Optimal Control, volumes 1 and 2 Athena Scientific, Belmont, Massachusetts.
[13] Blackwell, D. (1965) .Positive Dynamic Programming. Proceedings of the 5th Berkeley Symposium 3 415.428.
[14] Steven Skiena, Applications of Dynamic Programming, State University of New York Stony Brook, NY 11794–4400
[15] Bellman, Richard (1957): Dynamic Programming. Princeton University Press.
[16] Vijaya Ramachandran, “ALGORITHMS”, The University of Texas, Austin, Spring 2006.
[17] Professor Bergin, “Lecture Notes on Dynamic Programming”, Economics 200E, Spring 1998.
[18] Ross, Sheldon M. (1983): Introduction to Stochastic Dynamic Programming. San Diego: Academic Press.
[19] Oren Weimann, “Accelerating Dynamic Programming”, Department of Electrical Engineering and Computer Science, 2009.
[20] Traub, J.F. and Werschulz (1998) Complexity and Information Cambridge University Press, Cambridge, UK.
[21] David K. Smith and PASS Maths, “Dynamic programming: an introduction”, Millennium Mathematics Project, University of Cambridge, 1997−2009.
[22] Streufert, P. (1998) .Recursive Utility and Dynamic Programming. in S. Barbera, P. Hammond, and C. Seidl (editors) Handbook of Utility Theory, Volume 1 chapter 3 93.121.
[23] John W. Chinneck, Practical Optimization: a Gentle Introduction, 2006 http://www.sce.carleton.ca/faculty/chinneck/po.html
[24] George Hadley, “Nonlinear and Dynamic Programming, Addison-Wesley, 1962”
[25] Ronald A. Howard. , “Dynamic Programming and Markov Processes”, John Wiley & Sons, 1960
[26] Éva Tardos and Jon Kleinberg, “Algorithm Design”, Addison Wesley, 2005.
[27] Robert Sedgewick, Algorithms, ADDISON-WESLEY PUBLISHING COMPANY, August 1984.
[28] Herbert S. Wilf, Algorithms and Complexity, University of Pennsylvania , Internet Edition, Summer, 1994
[29] Nocedal, J., Wright, S. J.: Numerical Optimization, page 9, Springer, 2006.
