Apps Artificial Intelligence CSS DevOps Go JavaScript Laravel Linux MongoDB MySQL PHP Python Rust Svelte Vue

Working with CSV Files in Pandas

1 min read .
Working with CSV Files in Pandas

CSV (Comma-Separated Values) is a common format for storing and exchanging tabular data. Pandas makes it straightforward to export a DataFrame to CSV and load CSV data back into a DataFrame.

Install Pandas

If Pandas is not installed yet:

python -m pip install pandas

Using python -m pip helps ensure that pip belongs to the Python interpreter you intend to use.

Create a DataFrame

import pandas as pd

df = pd.DataFrame(
    {
        "Name": [
            "Braund, Mr. Owen Harris",
            "Allen, Mr. William Henry",
            "Bonnell, Miss. Elizabeth",
        ],
        "Age": [22, 35, 58],
        "Sex": ["male", "male", "female"],
    }
)

Export a DataFrame to CSV

Use DataFrame.to_csv():

df.to_csv("output.csv", index=False)

index=False prevents Pandas from writing the DataFrame index as an extra CSV column.

If you need an explicit encoding, UTF-8 is a common choice:

df.to_csv("output.csv", index=False, encoding="utf-8")

Read a CSV File

Load the file with pd.read_csv():

loaded_df = pd.read_csv("output.csv")
print(loaded_df)

Pandas infers column types in many ordinary datasets, but real-world CSV files may require options such as dtype, parse_dates, na_values, delimiter, or encoding.

Complete Example

import pandas as pd

df = pd.DataFrame(
    {
        "Name": [
            "Braund, Mr. Owen Harris",
            "Allen, Mr. William Henry",
            "Bonnell, Miss. Elizabeth",
        ],
        "Age": [22, 35, 58],
        "Sex": ["male", "male", "female"],
    }
)

df.to_csv("output.csv", index=False)
loaded_df = pd.read_csv("output.csv")
print(loaded_df)

Useful Options

Read only selected columns:

df = pd.read_csv("output.csv", usecols=["Name", "Age"])

Parse a date column while loading:

df = pd.read_csv("events.csv", parse_dates=["created_at"])

Read a large CSV in chunks:

for chunk in pd.read_csv("large.csv", chunksize=100_000):
    process(chunk)

Chunked reading can reduce peak memory usage when the entire dataset does not need to be loaded at once.

Conclusion

Pandas provides a simple CSV workflow through to_csv() and read_csv(). For small files the defaults are often enough; for production datasets, explicitly consider encoding, missing values, column types, date parsing, and memory usage.

Related Posts

chevron-up