map, filter, and reduce in Python with Examples

Python Streams

Python stream is a term for a particular paradigm for data processing that involves the sequential processing of data items as they pass through a pipeline of processes. Streams allow data processing to be continuous, effective, and memory-friendly without loading the entire dataset into memory at once.

The map, filter, and reduce functions in Python are higher-order functions that work on sequences of data, such as lists or other iterable objects. Streams can be used in conjunction with these methods. With the help of these functions, you may quickly and effectively analyze typical data on sequence elements.

What is functional programming?

Functional programming is a programming paradigm that treats computation as evaluating mathematical functions and avoids changing state and mutable data.

In functional programming, functions are first-class citizens, meaning they can be assigned to variables, passed as arguments to other functions, and returned as values from other functions.

Functional programming emphasizes immutability, meaning that once a value is assigned, it cannot be changed, and it avoids side effects, which are changes to the state or behaviour that affect the result of a function beyond its return value.

Functional programming is possible in Python using a number of features and tools, such as:

1. Higher-order functions - Python enables the assignment of functions to variables, the passing of functions as arguments, and the return of functions as values. Higher-order functions are the functions that accept other functions as arguments or return them as results and can be used as a result of this. Strong functional programming techniques like giving functions as parameters, returning functions from functions, and constructing functions on the fly are made possible by higher-order functions.

# Example of using higher-order functions in Python
# Function that adds 1 to the passed value (x)
def add(x):
    return x + 1

# Function that multiplies the passed value (x) by 2
def multiply(x):
    return x * 2

# Function that applies another function on passed value (x)
def apply(func, x):
    return func(x)

result1 = apply(add, 3)  # Result: 4
result2 = apply(multiply, 3)  # Result: 6

2. Lambda Functions - Lambda functions, also known as anonymous functions, can be defined inline without requiring a formal function declaration and are short and one-time-use functions. They are helpful for the performance of a single task that only requires one line of code to convey.

# Example of using lambda functions in Python
# Lambda function that adds 1 to x
add = lambda x: x + 1

# Lambda function that multiplies x by 2
multiply = lambda x: x * 2

result1 = add(3)  # Result: 4
result2 = multiply(3)  # Result: 6

3. map, filter, and reduce - Map, Filter, and Reduce are built-in Python functions that can be used for functional programming tasks. With the help of these operations, you may apply a specific function to sequence items using the 'map', filter sequence elements based on a condition using the 'filter', and cumulatively aggregate elements using the 'reduce'.

# Example of using map, filter, and reduce in Python
data = [1, 2, 3, 4, 5]

# Using the map to apply a function to each element
# Lambda function returns the square of x
result1 = map(lambda x: x * 2, data)  # Result: [2, 4, 6, 8, 10]

# Using the filter to filter elements based on a condition
# Lambda function returns True for an even number
result2 = filter(lambda x: x % 2 == 0, data)  # Result: [2, 4]

# Using reduce to aggregate elements
# Lambda function returns product of x and y
from functools import reduce
result3 = reduce(lambda x, y: x * y, data)  # Result: 120

4. List Comprehension - List comprehensions are supported by Python, which are simple and expressive techniques to build new lists from older ones. Functional programming techniques such as mapping, filtering, and aggregating can be implemented using list comprehensions.

# Example of using list comprehensions in Python
data = [1, 2, 3, 4, 5]

# Using list comprehension to apply a function to each element
result1 = [x * 2 for x in data]  # Result: [2, 4, 6, 8, 10]

# Using list comprehension to filter elements based on a condition
result2 = [x for x in data if x % 2 == 0]  # Result: [2, 4]

# Using list comprehension to aggregate elements
result3 = reduce(lambda x, y: x * y, data)  # Result 120

Let's dive deeper into the map(), filter(), and reduce() functions in Python.

1. map() - Python's map() method applies a specified function to each item of an iterable (such as a list, tuple, or string) and then returns a new iterable containing the results.

The map() syntax is as follows: map(function, iterable)

The first argument passed to the map function is itself a function, and the second argument passed is an iterable (sequence of elements) such as a list, tuple, set, string, etc.

Example 1 - usage of the map():

# Using map() to square each element of the data list
data = [1, 2, 3, 4, 5]

# Map function returns the map object
squares = map(lambda x: x*x, data)

# Iterating the elements of the squares
for i in squares:
    print(i, end=" ")

# Also, we can convert the map object into a list
squares = list(map(lambda x: x*x, data))
print(f"Squares: {squares}")

Output:

1, 4, 9, 16, 25
Squares: [1, 4, 9, 16, 25]

Here, the map function takes each element one by one from the data starting from x = 1. Each element is passed to the lambda function, returning its square. And the returned value is stored in the map object (an iterable).

Example 2:

As we all know that the input value is always a string. In the case of single-value input, we can use the 'int' function to convert it into an integer and the 'float' function to convert it into a float. But for a stream of inputs, we can use the map() function to achieve the same.

# Using map() to convert each element into an integer
data1 = list(map(int, input("Enter integer values: ").split()))

# Printing the data1 values
print(f"Integer data = {data1}")

# Using map() to convert each element into a float
data2 = list(map(float, input("Enter Floating values: ").split()))

# Printing the data2 values
print(f"Float data = {data2}")

Output:

Enter integer values: 24 5 6 84 2 6
Integer data = [24, 5, 6, 84, 2, 6]
Enter Floating values: 25.4 2.6 1.9 0.6 3 5
Float data = [25.4, 2.6, 1.9, 0.6, 3.0, 5.0]

Example 3: Passing multiple iterables

You are given two lists. One contains the base values, and the other contains the power value. You are supposed to find the x**y for each x in list one and y in list two.

# Python to demonstrate how to pass multiple iterables
# in the map function
# List that contains base values
base = [2, 4, 6, 8, 10]

# List that contains exponents
power = [1, 2, 3, 4, 5]

# Storing the list of x**y
answer = list(map(pow, base, power))

# Printing the result
print("Answer:", answer)

Output:

Answer: [2, 16, 216, 4096, 100000]

As we know, the pow(x, y) takes two arguments and returns the x**y. In the above example, the map function takes one element from base, and one element from power in order and passes them to the pow function till all the elements have been processed. In the end, it returns a map object, which is then type-casted into a list and stored in the variable named 'answer'.

2. filter() - The filter() function in Python filters elements from an iterable based on a given condition or function and returns a new iterable with the filtered elements.

The syntax for the filter() is as follows: filter(function, iterable)

Here also, the first argument passed to the filter function is itself a function, and the second argument passed is an iterable (sequence of elements) such as a list, tuple, set, string, etc.

Example 1 - usage of the filter():

You are given a list of integers and should filter the even numbers from the list.

# Using filter() to filter even numbers from a list
data = [1, 2, 3, 4, 5]

# The filter function filters the even numbers from the data
# and returns a filter object (an iterable)
evens = filter(lambda x: x % 2 == 0, data)

# Iterating the values of evens
for i in evens:
    print(i, end=" ")

# We can convert the filter object into a list as follows:
evens = list(filter(lambda x: x % 2 == 0, data))

# Printing the evens list
print(f"Evens = {evens}")

Output:

2 4
Evens = [2, 4]

Example 2: Filtering Perfect Squares

You are given a list of random integers and should filter the perfect squares from them. A perfect square is a number that can be expressed as the product of the same whole number.

# Python to demonstrate usage of filter function
from math import sqrt

# List that contains random integer values
data = [0, 1, 4, 6, 8, 9, 10, 12, 16, 81, 23, 36]

# Function that returns true for perfect squares
def isPerfectSqr(i):
    return sqrt(i).is_integer()

# Storing the result
answer = list(filter(isPerfectSqr, data))

# Printing the result
print("Answer: ", answer)

Output:

Answer:  [0, 1, 4, 9, 16, 81, 36]

In the above example, we have a data list that contains some random integers. The 'isPerfectSqr' function returns True for perfect square numbers and False for others. The filter function filters out the numbers from the data, which is a perfect square, and returns an iterable that contains those perfect square numbers. In the end, we converted the result into a list and printed it into the console.

Example 3: Filter Out Names Starting with H

You are given a list containing names of persons and you should filter out the names starting with the letter 'H'. Below is the solution of the problem:

# Method 1

# Using filter() to filter names starting with letter H
# A list containing names
names = ["Arun", "Sonu", "Harsh", "Harry", "Anu", "Hassi"]

# The filter function filters the name from the names
# and returns a filter object (an iterable)
# We can convert the filter object into a list as follows:
name_start_with_H = list(filter(lambda x: x[0] == 'H', names))

# Printing the name_start_with_H list
print(f"Method 1 result = {name_start_with_H}")

# Method 2

# We can also use a function instead of lambda function
# The H_name function returns true if x (name) starts with 'H'
def H_name(x):
    return x[0] == 'H'

# Filtering the result and printing it into the console
name_start_with_H = list(filter(H_name, names))
print(f"Method 2 result = {name_start_with_H}")

Output:

Method 1 result = ['Harsh', 'Harry', 'Hassi']
Method 2 result = ['Harsh', 'Harry', 'Hassi']

In the above example, the lambda or H_name function returns true for each x in names starting with letter H. The filter function filters all the names which satisfies the condition.

3. reduce() - In Python, reduce() is a built-in function that applies a given function to the elements of an iterable, reducing them to a single value.

The syntax for reduce() is as follows: reduce(function, iterable[, initializer])

The function argument is a function that takes two arguments and returns a single value. The first argument is the accumulated value, and the second argument is the current value from the iterable.

The iterable argument is the sequence of values to be reduced.

The optional initializer argument is used to provide an initial value for the accumulated result. If no initializer is specified, the first element of the iterable is used as the initial value.

Here's an example that demonstrates how to use reduce() to find the sum of a list of numbers:

Example 1:

You are given a list containing some integers and you should find the sum of all elements in the list using reduce function. Below is the solution of the problem:

# Examples to understand the reduce() function
from functools import reduce

# Function that returns the sum of two numbers
def add(a, b):
    return a + b

# Our Iterable
num_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# add function is passed as the first argument, and num_list is passed as the second argument
sum = reduce(add, num_list)
print(f"Sum of the integers of num_list : {sum}")

# Passing 10 as an initial value
sum = reduce(add, num_list, 10)
print(f"Sum of the integers of num_list with initial value 10 : {sum}")

Output:

Sum of the integers of num_list : 55
Sum of the integers of num_list with initial value 10 : 65

In the above example, the reduce function takes two elements 1 and 2 from the num_list and passes to the add function in the first iteration. The add function returns the some of the 1 and 2 which is 3. In the second iteration, the reduce function passes the result of the previous call which is 3 and the next element which is also 3. This process is repeated until all elements have been processed.

In the case, where we pass the initial value (10), the reduce function takes one element from the num_list and initial value (10) and passes to the add function in the first iteration.

Example 2: Using operator functions with reduce function

In the below example, we have used operator.add to perform addition, operator.mul to perform multiplication and operator.concat to perform concatenation on strings.

# Python program to demonstrate
# how to use operator functions with reduce function

# Importing reduce function
from functools import reduce

# Importing operator
import operator

# Creating lists
my_list1 = [1, 2, 3, 4, 5]
my_list2 = ["I", "Love", "Javatpoint"]

# Calculating the sum of the numbers of my_list1
# using reduce and operator.add
sum = reduce(operator.add, my_list1)

# Calculating the product of the numbers of my_list1
# using reduce and operator.mul
product = reduce(operator.mul, my_list1)

# Concatenating all the elements in my_list2
# using reduce and operator.concat
concated_str1 = reduce(operator.concat, my_list2)

# We can achieve the same output by using operator.add
concated_str2 = reduce(operator.add, my_list2)

# Printing result
print(f"Sum of all elements in my_list1 : {sum}")
print(f"Product of all elements in my_list1 : {product}")
print(f"Concatenated string by using operator.concat : {concated_str1}")
print(f"Concatenated string by using operator.add : {concated_str2}")

Output:

Sum of all elements in my_list1 : 15
Product of all elements in my_list1 : 120
Concatenated string by using operator.concat : ILoveJavatpoint
Concatenated string by using operator.add : ILoveJavatpoint

Here, the function argument in the reduce function is replaced with the operator functions. All the steps are same as previous examples.

Example 3: Find the largest of the given numbers.

In this example, you are given a list of integers and you should find the largest number using the reduce function. Below is the solution of the problem:

The are three methods, we can use to achieve the same result:

Method 1 - Using normal function

# Importing reduce function from the functools module
from functools import reduce

# A list containing some integers
num = [20, 22, 24, 12, 6, 88, 10, 55, 66]

""" Method 1 - Using simple function """

# Function that returns the largest of x and y
def large(x, y):
    return x if x > y else y

# Using reduce to find the largest of all and printing the result
largest = reduce(large, num)
print(f"Largest found with method 1: {largest}")

Method 2 - Using Lambda function

""" Method 2 - Using lambda function """

# Using reduce to find the largest of all and printing the result
largest = reduce(lambda x, y: x if x > y else y, num)
print(f"Largest found with method 2: {largest}")

Method 3 - Using max function

""" Method 3 - Using max() function """

# Using reduce to find the largest of all and printing the result
largest = reduce(max, num)
print(f"Largest found with method 3: {largest}")

Output:

Largest found with method 1: 88
Largest found with method 2: 88
Largest found with method 3: 88

In the above example, the large function, lambda function, and the max function returns the maximum of x and y to the reduce function. And the reduce function parse all the elements one by one. At the end, it returns the largest of all.

CONCLUSION:

In conclusion, map(), filter(), and reduce() are built-in functions in Python that are commonly used for functional programming.

map() is used to apply a given function to each element of an iterable and returns a new iterable with the results.
filter() is used to filter elements from an iterable based on a given condition or function and returns a new iterable with the filtered elements.
reduce() is used to apply a given function to the elements of an iterable in a cumulative way, reducing the iterable to a single value.

These functions or tools give you strong capabilities for processing data quickly and expressively, making it simple to convert, filter, and aggregate data. They are frequently used in Python's functional programming concepts to create readable and efficient codes.

Next TopicEdit Distance in Python

← prev next →