Lemmatization and Tokenize with TextBlobLemmatization is the process of joining the different inflected terms to be considered as one thing. Lemmatization is similar to stemming. However, it offers contextual meaning to the terms. It also links words that share the same meaning and are considered one word. Text pre-processing includes stemming and Lemmatization. There are instances when people get confused with the two terms, and they are often viewed as being the same. However, Lemmatization is more advantageous than stemming because it permits the study of morphology in words. Applications of Lemmatization:
Examples of lemmatization: The main difference between stemming and lemmatizing is that lemmatize requires the speech part parameters, "pos" If not provided, the default will be "noun." Below is the way to implement lemmatization with TextBlob. Code: Output: painters: painter birds: bird worst: bad Tokenize Text using TextBlobTextBlob module is a Python library that offers an API that is simple to use its methods and perform simple NLP tasks. This module is developed on the base of the NLTK module. Install TextBlob by using the following commands on the terminal: It will then enable TextBlob in addition to downloading the needed NLTK corpora. The above process can take a long time due to a large number of tokenizers, chunkers, various algorithms, and the entire corpus to download. The terms that are commonly used include:
Code: Output: Word Tokenize from paragraph: ['There', 'were', 'three', 'friends', 'name', 'Jemmy', 'Jacky', 'Kenny', 'They', 'have', 'been', 'friends', 'forever', 'since', 'pre', 'school.but', 'somehow', 'Jemmy', "'s", 'bestfriend', 'is', 'Jacky', 'and', 'whenever', 'jemmy', 'and', 'kenny', 'lefts', 'alone', 'they', 'endup', 'being', 'quite.One', 'day', 'they', 'all', 'decided', 'to', 'plan', 'a', 'trip', 'together', 'after', 'graduation', 'They', 'all', 'went', 'to', 'KashmireKashmire', 'trip', 'was', 'really', 'good', 'they', 'all', 'created', 'lifetime', 'memories', 'together', 'After', 'that', 'trip', 'they', 'have', 'to', 'focus', 'on', 'there', 'future', 'Which', 'stream', 'they', 'have', 'to', 'choose', 'and', 'career', 'path', 'they', 'should', 'choose', 'for', 'future'] Sentence Tokenize from paragraph: [Sentence("There were three friends name, Jemmy, Jacky, Kenny."), Sentence("They have been friends forever since pre school.but somehow Jemmy's bestfriend is Jacky and whenever jemmy and kenny lefts alone, they endup being quite.One day they all decided to plan a trip together after graduation."), Sentence("They all went to KashmireKashmire trip was really good, they all created lifetime memories together."), Sentence("After that trip they have to focus on there future."), Sentence("Which stream they have to choose and career path they should choose for future.")] Next TopicHow to Round Numbers in Python |
We provides tutorials and interview questions of all technology like java tutorial, android, java frameworks
G-13, 2nd Floor, Sec-3, Noida, UP, 201301, India