Combining Multiple CSV Files into a Single DataFrame using Python's Glob Module

Python

Note that all csv files must contain the same columns for this to work.

 1|  import glob
 2|  
 3|  # 1. Declare directory where csv files are stored
 4|  directory = "../data/folds/*.csv"
 5|  
 6|  # 2. Create empty dataframe
 7|  df = pd.DataFrame()
 8|  
 9|  # 3. Iterate through each csv file in the directory, read it into a
10|  # temporary dataframe and then append this to dataframe to the main
11|  # dataframe
12|  for file in glob.glob(directory):
13|      temp_df = pd.read_csv(file)
14|      df = df.append(temp_df)
detro
Pandas | Csv | Glob
Did you find this snippet useful?

Sign up for free to to add this to your code library

Remove Stop Words from Text in DataFrame Column
Python

Pandas | Nltk | Nlp

149
LightGBM Custom Loss Function
Python

LightGBM | Loss function

132
Tuning XGBoost Hyperparameters with Grid Search
Python

XGBoost | Tuning

127
How to Convert DataFrame Values Into Percentages
Python

Lambda | Percentages | Pipe

119
How to Scale Data Using Standard Scaler But Keep Column Names
Python

Scaler | Standard

92
LightGBM Hyperparameter Tuning with GridSearch
Python

Python | LightGBM | Hyperparameter tuning | Gridsearch

91
Calculating Root Mean Squared Error (RMSE) with Sklearn and Python
Python

Rmse

86
Dynamically Create Columns in Pandas Dataframe
Python

Pandas | Columns

68
How to Train a Catboost Classifier with GridSearch Hyperparameter Tuning
Python

Catboost | Hyperparameter tuning

68
Overwriting a Python Logging File Instead of Appending
Python

Logging

65