Apache Pig TOKENIZE Function
The Apache Pig TOKENIZE function is used to splits the existing string and generates a bag of words in a result.
Example of TOKENIZE Function
In this example, we split the string in the tokens.
Steps to execute TOKENIZE Function
- Create a text file in your local machine and insert the list of tuples.
- Check the tuples inserted in the text files.
- Upload the text files on HDFS in the specific directory.
- Open the pig MapReduce run mode.
- Load the file that contains the data.
- Now, execute and verify the data.
- Let's return the tokenized form of the string.
Here, we got the desired output.