Apache Pig FILTER Operator

The Apache Pig FILTER operator is used to remove duplicate tuples in a relation. Initially, Pig sorts the given data and then eliminates duplicates.

Example of FILTER Operator

In this example, we eliminate duplicate tuples.

Steps to execute FILTER Operator

• Create a text file in your local machine and provide some values to it.

• Check the values written in the text files.

• Upload the text files on HDFS in the specific directory.

• Open the pig MapReduce run mode.

• Load the file that contains the data.

• Now, execute and verify the data

• Let's execute FILTER operator to eliminate duplicate tuples.

• Now, execute and verify the data.

Here, we got the desired output.