Natural Language Processing

Text cleaning, text analysis, NLU, and NLG.
Analyzing and automating humanity's greatest invention.

Modules

KDNuggets Text Cleaning

Reviewing options and best practices for cleaning raw text.
Preparing online articles' text for analysis and topic modeling.

  • Text cleaning, lemmatization, tokenization, Tf-idf
  • NLTK and regex for text cleaning
  • TextBlob, Stanford CoreNLP, Gensim for analysis & text preparation

Topic Modeling & Tag Prediction

Unsupervised topic modeling & supervised tag prediction
Comparing generated topics with real and predicted article tags

  • Latent Dirichlet Allocation (LDA) with Gensim for topic modeling
  • Evaluation of use-cases for Bayesian approaches vs. deep learning

SummaryBot

Coming Soon...
Chat bot with consistent personality and the function of sumarizing textual input

  • Pre-programmed (through RL) personality
  • Summarizes content in text or webpage provided
  • Simply googles answers to queries it doesn't know