Javatpoint Logo
Javatpoint Logo

Spark Intersection Function

In Spark, Intersection function returns a new dataset that contains the intersection of elements present in the different datasets. So, it returns only a single row. This function behaves just like the INTERSECT query in SQL.

Example of Intersection function

In this example, we intersect the elements of two datasets.

  • To open the Spark in Scala mode, follow the below command.
Spark Intersection Function
  • Create an RDD using the parallelized collection.
  • Now, we can read the generated result by using the following command.
Spark Intersection Function
  • Create another RDD using parallelized collection.
  • Now, we can read the generated result by using the following command.
Spark Intersection Function
  • Apply intersection() function to return the intersection of the elements.
  • Now, we can read the generated result by using the following command.
Spark Intersection Function

Here, we got the desired output.






Help Others, Please Share

facebook twitter google plus pinterest

Learn Latest Tutorials


Preparation


Trending Technologies


B.Tech / MCA