How to Return the Most Frequent Bigrams from Text Using NLTK

Python

In this snippet we return one bigram that appears at least twice in the string variable text.

 1|  import nltk
 2|  from nltk.collocations import *
 3|  bigram_assoc_measures = nltk.collocations.BigramAssocMeasures()
 4|  
 5|  text = 'One Two One Two Three Four Five Six'
 6|  
 7|  #1. Split text into words
 8|  text = text.split()
 9|  
10|  #2. Set minimum number of bigrams to extract and 
11|  #of those how many to return
12|  minimum_number_of_bigrams = 2
13|  top_bigrams_to_return = 1
14|  
15|  #3. Get bigrams contained in text variable
16|  finder = BigramCollocationFinder.from_words(text)
17|  
18|  #4. Filter bigrams to those that appear at least twice
19|  finder.apply_freq_filter(minimum_number_of_bigrams) 
20|  
21|  #5. Return one of the top bigrams
22|  finder.nbest(bigram_assoc_measures.pmi, bigrams_to_return)
analyseup
Nltk | Nlp | Bigrams
Did you find this snippet useful?

Sign up for free to to add this to your code library

Remove Stop Words from Text in DataFrame Column
Python

Pandas | Nltk | Nlp

133
LightGBM Custom Loss Function
Python

LightGBM | Loss function

121
Tuning XGBoost Hyperparameters with Grid Search
Python

XGBoost | Tuning

116
How to Convert DataFrame Values Into Percentages
Python

Lambda | Percentages | Pipe

109
How to Scale Data Using Standard Scaler But Keep Column Names
Python

Scaler | Standard

78
LightGBM Hyperparameter Tuning with GridSearch
Python

Python | LightGBM | Hyperparameter tuning | Gridsearch

77
Calculating Root Mean Squared Error (RMSE) with Sklearn and Python
Python

Rmse

77
Dynamically Create Columns in Pandas Dataframe
Python

Pandas | Columns

61
How to Train a Catboost Classifier with GridSearch Hyperparameter Tuning
Python

Catboost | Hyperparameter tuning

60
Overwriting a Python Logging File Instead of Appending
Python

Logging

55