NLTK and Natural Language Processing
These files serve as an introduction to the Natural Language Toolkit.
DEMO FILE: nltk-test.py After installing the required libraries from requirements.txt, this file may be run to demonstrate rudimentary text and tokenization methods in nltk, including COLLOCATIONS, CONCORDANCE, and SIMILAR. Sample frequency distribution plots are also produced (matplotlib required).
SAMPLE TEXT: murakami-norwood.txt An excerpt from Harumi Murakami's book "Norwegian Wood", typed by the author of this repository.
STEMMING: stemming.py Example of performing stemming.
STOP WORDS: stop-words.py Example of filtering out stop words (meaningless words) from sentence in file.
TAGGING: tagging.py Example of Part-of-Speech tagging using text from 2005 and 2006 State of the Union addresses (nltk corpus).
CHUNKING: chunking.py Example of chunking tokens using text from 2005 and 2006 State of the Union addresses (nltk corpus).