Dynamic programming vs Backtracking

Before understanding the differences between dynamic programming and backtracking, we should know about dynamic programming and backtracking separately.

What is Backtracking?

Backtracking is one of the problem-solving techniques. Using this technique, we can solve our problem. This strategy uses a Brute force approach, and the brute force approach says that for the given problem, we should try out all the possible solutions and pick the desired solution from all the possible solutions. In contrast, dynamic programming is used to solve the optimization problem, but backtracking is not used to solve the optimization problems. When multiple solutions to a given problem exist, then backtracking uses all the solutions to solve the problem.

Now we will understand that how dynamic programming uses the brute force approach through an example.

Suppose there are three are three students, two are girls, and one is a boy. We have three chairs, and we have to arrange these students in these chairs. In how many ways we can arrange these students?. As there are 3 students, we can arrange these students in 3! ways, i.e., 6 ways. Now, we have to find out all the possible arrangements and all the arrangements or solutions can be represented in the form of a tree known as a state-space tree.

G1, G2, B1

State-space tree:

First arrangement

First, consider a node shown as below:

Consider the first girl G1 in the first level, i.e., first chair shown as below:

Consider the second girl G2 in the next level, i.e., second chair shown as below:

Consider a boy B1 in the next level, i.e., third chair shown as below:

Second arrangement

We will backtrack. First, we remove B1 from the third chair and G2 from the second chair. Add B1 at the second chair and then G2 at the third chair shown as below:

Third arrangement

We will backtrack again. First, we remove G2 from the third chair, then we remove B1 from the second chair and then we remove the G1 from the first chair. Consider the G2 in the first chair shown as below:

Consider the G1 in the next level, i.e., second chair shown as below:

Consider the B1 in the third level, i.e., third chair shown as below:

4th arrangement

We will backtrack again. First, we remove B1 from the third chair then we remove G1 from the second chair. Add B1 at the second chair and then G1 at the third chair shown as below:

5th arrangement

We will backtrack again. First, we remove G1 from the third chair then we remove B1 from the second chair and then we remove G2 from the first chair.

Consider B1 in the first level, i.e., first chair shown as below:

Consider G1 in the next level, i.e., second chair shown as below:

Consider G2 in the next level, i.e., third chair shown as below:

6th arrangement

We will backtrack again. First, we remove G2 from the third chair and then we remove G1 from the second chair. Add G2 at the second chair and then add G1 at the third chair shown as below:

What is Dynamic Programming?

Dynamic programming is a technique for solving certain type of complex problems efficiently by breaking them down into simpler subproblems and solving each problem exactly once. Dynamic programming stores the result of a subproblem in a table and reuse them when needed to avoid solving the same problems again and again.

What type of problems can be solved using dynamic programming?

The following are the two problems that can be solved using dynamic programming:

• Optimal substructure: A given problem has optimal substructure property if optimal solution of the given problem can be obtained using the optimal solutions of subproblems. In other words, we can define the solution to the problem by using a recurrence relationship based on its subproblems.
• Overlapping subproblems: A given problem has overlapping subproblems property if to solve the problem we have to solve its subproblems multiple times.

Using dynamic programming approach, we avoid solving overlapping subproblems and we solve each problem only once and save the result in a cache. When its needed again we will get the result from the cache instead of solving them again.

Approaches of Dynamic programming

There are two approaches used to implement the dynamic programming:

• Top-down approach: It is also known as a memoization. It is implemented using both recursion and caching. Whenever the recursive function is called, we check the cache to see whether the problem has already been solved. If it is already solved then we return the result from the cache else we will solve the subproblem, save the result into cache and return the result.
• Bottom-up approach: It is also known as tabulation. This technique is used to solve all the smaller subproblems first then move to the larger subproblem that use the result of the smaller subproblems.

Differences between Dynamic programming and Backtracking

• Dynamic programming is a technique of dividing the complex problem into simpler sub-problems. This technique is applicable to the problems that exhibit the following properties:
  Overlapping subproblems
  Optimal sub-structure
  Backtracking is a technique that recursively build a solution incrementally and remove all the solutions that does not satisfy the constraints of the problem at any point of time.
• The major difference between the dynamic programming and backtracking is that the dynamic programming completely relies on the principle of optimality which means that the sub sequence of a sequence should be optimal. In contrast to dynamic programming, backtracking does not guarantee the complete optimal solution.
• Dynamic programming is a technique that solves the optimization problem. Optimization problem uses either minimum or maximum result. In contrast to dynamic programming, backtracking uses the brute force approach without considering the optimization problem. If we have multiple solutions then it considers all those solutions.