Javatpoint Logo
Javatpoint Logo

Spark Char Count Example

In Spark char count example, we find out the frequency of each character exists in a particular file. Here, we use Scala language to perform Spark operations.

Steps to execute Spark char count example

In this example, we find and display the number of occurrences of each character.

  • Create a text file in your local machine and write some text into it.
Spark Char Count Example
  • Check the text written in the sparkdata.txt file.
Spark Char Count Example
  • Create a directory in HDFS, where to kept text file.
  • Upload the sparkdata.txt file on HDFS in the specific directory.
Spark Char Count Example
  • Now, follow the below command to open the spark in Scala mode.
Spark Char Count Example
  • Let's create an RDD by using the following command.

Here, pass any file name that contains the data.

  • Now, we can read the generated result by using the following command.
Spark Char Count Example
  • Here, we split the existing data in the form of individual words by using the following command.
  • Now, we can read the generated result by using the following command.
Spark Char Count Example
  • Now, perform the map operation.

Here, we are assigning a value 1 to each word.

  • Now, we can read the generated result by using the following command.
Spark Char Count Example
  • Now, perform the reduce operation

Here, we are summarizing the generated data.

  • Now, we can read the generated result by using the following command.
Spark Char Count Example

Here, we got the desired output.


Next Topic#




Help Others, Please Share

facebook twitter google plus pinterest

Learn Latest Tutorials


Preparation


Trending Technologies


B.Tech / MCA