Spark Word Count Example

In Spark word count example, we find out the frequency of each word exists in a particular file. Here, we use Scala language to perform Spark operations.

Steps to execute Spark word count example

In this example, we find and display the number of occurrences of each word.

• Create a text file in your local machine and write some text into it.

• Check the text written in the sparkdata.txt file.

• Create a directory in HDFS, where to kept text file.

• Upload the sparkdata.txt file on HDFS in the specific directory.

• Now, follow the below command to open the spark in Scala mode.

• Let's create an RDD by using the following command.

Here, pass any file name that contains the data.

• Now, we can read the generated result by using the following command.

• Here, we split the existing data in the form of individual words by using the following command.

• Now, we can read the generated result by using the following command.

• Now, perform the map operation.

Here, we are assigning a value 1 to each word.

• Now, we can read the generated result by using the following command.

• Now, perform the reduce operation

Here, we are summarizing the generated data.

• Now, we can read the generated result by using the following command.

Here, we got the desired output.