Knapsack Problem in Python

In this tutorial, we will learn how to solve the Knapsack problem in Python. We will use several approaches find the solution. In this problem, we have a set of N items, each with its weight and value. Our goal is to select the items and place them in a knapsack with a limited capacity, denoted as W, in a way that maximizes the total value of the items in the knapsack. It's important to mention that we can choose each item only once.

In other words, we are provided with two arrays: one containing values (val) and the other representing the weights (wt) of N items. Additionally, there's an integer W, signifying the maximum weight the knapsack can hold. Our task is to determine the most valuable combination of items from the val array, while ensuring that the sum of the corresponding weights in the wt array doesn't exceed the knapsack's capacity. It's important to note that you can't split an item; you either include it entirely in your selection or leave it out. This problem is often referred to as the 0-1 knapsack problem.

Example 1:

Input

N = 4
W = 6
Values[] = {5,7,2,4}
Weights[] = {3,4,5,2}

Output

Explanation: To maximize the value within the knapsack's capacity of 6 units, select the first item with a value of 5 and the fourth item with a value of 4.

Example 2:

Input

N = 5
W = 7
Values[] = [3,8,6,2,5]
Weights[] = [2,3,4,7,1]

Output

Explanation: Choose the second and fifth items to obtain the maximum total value of 12 while staying within the knapsack's capacity.

Let's solve the above problem using the various approaches.

Approach - 1: Brute-Force Approach

First, we will solve the problem using the brute-force approach. Solving the 0/1 knapsack problem using this approach involves generating all possible combinations of items and calculating the total value for each combination. Then we choose the combination with the maximum value that doesn't exceed the knapsack's capacity. Here's a Python implementation of this brute-force approach:

Example -

def knapsack_bruteforce(values, weights, capacity):
    n = len(values)
    
    # Function to calculate the total value of a combination
    def get_total_value(combination):
        total_value = 0
        total_weight = 0
        for i in range(n):
            if combination[i] == 1:
                total_value += values[i]
                total_weight += weights[i]
        return total_value, total_weight
    
    # Initialize variables to keep track of the best combination and its value
    best_combination = [0] * n
    best_value = 0
    
    # Generate all possible combinations using binary representation
    for i in range(2**n):
        combination = [int(bit) for bit in bin(i)[2:].zfill(n)]
        total_value, total_weight = get_total_value(combination)
        
        # Check if this combination is feasible and has a higher value
        if total_weight <= capacity and total_value > best_value:
            best_combination = combination
            best_value = total_value
    
    return best_value, best_combination

# Example usage
values = [1, 2, 3]
weights = [4, 5, 1]
capacity = 4
max_value, best_combination = knapsack_bruteforce(values, weights, capacity)
print("Maximum Value:", max_value)
print("Selected Items:", best_combination)

Output

Maximum Value: 3
Selected Items: [0, 0, 1]

Explanation -

Let's understand the following steps of the code.

First, we define a knapsack_bruteforce() function that takes three arguments: values, weights, and capacity. These arguments represent the values and weights of items, and the capacity of the knapsack.
Inside the function, it calculates the length of the `values` list (number of items) and assigns it to the variable n.
Then we define get_total_value() function inside the main function. This inner function calculates the total value and total weight of a combination of items, represented by a binary list.
It initializes variables best_combination as a list of zeros (indicating no items selected) and best_value as 0 to keep track of the best combination found so far.
Then it enters a loop that generates all possible combinations of items. It uses a binary representation of numbers from 0 to 2^n (where n is the number of items) to indicate item selections.
For each combination, it calls the get_total_value() function to compute the total value and total weight of that combination.
It checks whether the combination is feasible (total weight doesn't exceed the knapsack capacity) and if it has a higher total value than the best combination found so far.
If these conditions are satisfied, it updates best_combination and best_value with the current combination and its total value.
After checking all possible combinations, the function returns the maximum total value (`best_value`) and the corresponding combination (`best_combination`) of selected items.

Remember that the brute-force approach is inefficient for larger instances of the knapsack problem due to its exponential time complexity. To overcome it, we will use the dynamic approach for efficient approach.

Approach - 2: Dynamic Approach

Let's understand the following example -

Example -

def knapsack_max_value(capacity, weights, values):
    n = len(values)
    dp = [[0 for _ in range(capacity + 1)] for _ in range(n + 1)]

    for i in range(n + 1):
        for w in range(capacity + 1):
            if i == 0 or w == 0:
                dp[i][w] = 0
            elif weights[i - 1] <= w:
                dp[i][w] = max(values[i - 1] + dp[i - 1][w - weights[i - 1]], dp[i - 1][w])
            else:
                dp[i][w] = dp[i - 1][w]

    return dp[n][capacity]
# Example usage
values = [1, 2, 3]
weights = [4, 5, 1]
capacity = 4
max_value = knapsack_max_value(capacity, weights, values)
print("Maximum Value:", max_value)

Output

Maximum Value: 3

Explanation -

Let's understand the following code of 0/1 knapsack problem using dynamic programming.

The function knapsack_max_value() is defined to find the maximum value that can be obtained within the given knapsack capacity and it takes the three arguments - The first argument is capacity which represents the maximum weight the knapsack can hold. The second argument is a weight which represents a list of item weights and the values represents a list of item values.
The variable n is initialized to the length of the values list, representing the number of items.
A 2D list dp is created to store the maximum values achieved for different combinations of items and capacities. It has (n + 1) rows and (capacity + 1) columns.
The dynamic programming algorithm builds the dp table in a bottom-up manner, considering all possible combinations of items and capacities.
Two nested loops iterate over the number of items (i) and the available capacity (w). For each combination:
1. If either there are no items left (`i == 0`) or the knapsack has no capacity left (`w == 0`), the maximum value is 0.
2. If the weight of the current item (`weights[i-1]`) is less than or equal to the available capacity (`w`), the algorithm calculates the maximum value by considering two options: including the current item and excluding it.
3. If the weight of the current item is greater than the available capacity, the maximum value is the same as the maximum value achieved without this item.
The function returns the maximum value obtained at the end of the dynamic programming process, which is stored in `dp[n][capacity]`.
In the example usage, the code demonstrates how to use the knapsack_max_value() function with a sample set of values, weights, and knapsack capacity. It prints the maximum value that can be achieved within the given constraints.

The dynamic programming approach efficiently solves the 0/1 knapsack problem by considering all possible combinations and selecting the one with the maximum value while respecting the knapsack's capacity.

Approach - 3: Greedy Approach

Greedy algorithms are a technique used to solve optimization problems, where the goal is to discover the best possible solution based on a specific set of criteria. These algorithms make decisions that seem to be the most beneficial at each step, with the expectation that these locally optimal choices will ultimately lead to the most favorable overall solution.

Example -

def knapsack_greedy(values, weights, capacity):
    n = len(values)
    value_per_weight = [(values[i] / weights[i], values[i], weights[i]) for i in range(n)]
    # Sort items by value-to-weight ratio in descending order
    value_per_weight.sort(reverse=True)

    total_value = 0
    selected_items = [0] * n

    for i in range(n):
        if capacity == 0:
            break
        if value_per_weight[i][2] <= capacity:
            total_value += value_per_weight[i][1]
            capacity -= value_per_weight[i][2]
            selected_items[i] = 1

    return total_value, selected_items
values = [1, 2, 3]
weights = [4, 5, 1]
capacity = 4
max_value, best_combination = knapsack_greedy(values, weights, capacity)
print("Maximum Value (Greedy Approach):", max_value)
print("Selected Items:", best_combination)

Output

Maximum Value (Greedy Approach): 3
Selected Items: [1, 0, 0]

Conclusion

In this tutorial, we explored different ways to tackle the 0/1 Knapsack problem in Python. We discussed the brute-force approach, which systematically generates all possible combinations of items and the dynamic programming approach which efficiently computes the optimal solution using a 2D table. The greedy approach, which makes locally optimal choices based on value-to-weight ratios.

The dynamic programming approach is the most efficient, while the greedy approach provides a quick but not always optimal solution. The choice of method depends on problem size and the need for an optimal or heuristic solution.

Next TopicMerge-two-sorted-arrays-without-extra-space

← prev next →