Using Selectors in Beautiful Soup to Scrape Using Classes & IDs

Python

Here we use a Beautiful Soup selector to get all the tags in a webpage with that uses the css class "links" and also the tag in the webpage with the css id "main_header".

 1|  import requests
 2|  from bs4 import BeautifulSoup
 3|  
 4|  response = requests.get('https://en.wikipedia.org/wiki/FTSE_100_Index')
 5|  parser = BeautifulSoup(response.content, 'html.parser')
 6|  
 7|  links = parser.select(".links")
 8|  
 9|  main_header = parser.select("#main_header")
detro
Beautiful soup | Scarping
Did you find this snippet useful?

Sign up for free to to add this to your code library

Remove Stop Words from Text in DataFrame Column
Python

Pandas | Nltk | Nlp

133
LightGBM Custom Loss Function
Python

LightGBM | Loss function

120
Tuning XGBoost Hyperparameters with Grid Search
Python

XGBoost | Tuning

116
How to Convert DataFrame Values Into Percentages
Python

Lambda | Percentages | Pipe

109
How to Scale Data Using Standard Scaler But Keep Column Names
Python

Scaler | Standard

78
Calculating Root Mean Squared Error (RMSE) with Sklearn and Python
Python

Rmse

77
LightGBM Hyperparameter Tuning with GridSearch
Python

Python | LightGBM | Hyperparameter tuning | Gridsearch

77
Dynamically Create Columns in Pandas Dataframe
Python

Pandas | Columns

60
How to Train a Catboost Classifier with GridSearch Hyperparameter Tuning
Python

Catboost | Hyperparameter tuning

60
Overwriting a Python Logging File Instead of Appending
Python

Logging

55