Everything about pandas,
in one page.
What pandas is, why it exists, its core types, every major topic with runnable examples, a condensed cheat sheet, diagrams of how a DataFrame is put together - and a one-click download of the whole thing as notes.
What is pandas?
pandas is an open-source Python library for working with structured (tabular) data — think spreadsheets or SQL tables, but manipulated with code. It introduces two core objects, Series (a labeled column) and DataFrame (a labeled table), and a large, consistent API for reading, cleaning, reshaping, combining, and summarizing that data. It's built directly on top of NumPy, so the heavy lifting runs at C speed instead of interpreted Python loops.
Why use it
- Pandas gives Python a fast, labeled, two-dimensional table (DataFrame) — the same mental model as a spreadsheet or SQL table, but scriptable.
- It is built on NumPy, so bulk operations run in compiled C loops instead of slow Python for-loops.
- One library covers the whole data-wrangling lifecycle: read files, clean, reshape, merge, aggregate, and write results back out.
- It integrates directly with the rest of the PyData stack — NumPy, Matplotlib, scikit-learn, SQLAlchemy — so a DataFrame can flow straight into a model or a chart.
- Labeled axes (row index + column names) mean you select data by meaning ('sales in March') instead of by raw position.
Why it's needed
- Raw Python lists/dicts have no concept of alignment — combining two datasets means writing your own matching logic. Pandas aligns on the index automatically.
- Real-world data is messy: missing values, mixed types, duplicate rows, inconsistent dates. Pandas has dedicated, tested tools (isna, dropna, fillna, drop_duplicates, to_datetime) for exactly this.
- Analysts constantly need to group-and-summarize ('total revenue per region per month'). groupby + agg does this in one readable line instead of nested loops.
- Without pandas you would hand-roll CSV/Excel/JSON/SQL parsing every time. Pandas standardizes all of it behind read_*/to_* functions.
- Vectorized operations (column + column, column.str.upper(), etc.) are both shorter to write and orders of magnitude faster than looping row by row.
Core types
Series
1-D labeled array. One column of data + an index.
DataFrame
2-D labeled table. A dict of Series sharing one index.
Index
The immutable label array attached to rows (and columns).
Sketch — anatomy of a DataFrame
Installation
pip install pandas
# inside a notebook / script
import pandas as pd
print(pd.__version__)Typical workflow
Detailed notes — every topic
- pd.Series(data, index=...) builds a 1-D labeled array.
- pd.DataFrame(data) accepts dicts of lists, lists of dicts, NumPy arrays, or another DataFrame.
- Every axis (rows and columns) is labeled — that label set is the 'Index'.
import pandas as pd
s = pd.Series([10, 20, 30], index=["a", "b", "c"], name="scores")
# a 10
# b 20
# c 30data = {
"city": ["Pune", "Delhi", "Goa"],
"temp_c": [29, 34, 31],
}
df = pd.DataFrame(data)
# city temp_c
# 0 Pune 29
# 1 Delhi 34
# 2 Goa 31Give DataFrame a meaningful index up front (index='city' or set_index later) — it makes every later lookup read like plain English.
Formula / method cheat sheet
Create / Inspect
| pd.Series(data, index=...) | 1-D labeled array |
| pd.DataFrame(data) | 2-D labeled table |
| df.head(n) / df.tail(n) | First / last n rows |
| df.info() | Dtypes + non-null counts |
| df.describe() | Summary statistics |
| df.shape / df.dtypes | Dimensions / column types |
I/O
| pd.read_csv(path) | Load a CSV into a DataFrame |
| df.to_csv(path, index=False) | Write DataFrame to CSV |
| pd.read_excel(path, sheet_name=) | Load an Excel sheet |
| pd.read_json(path) | Load JSON records |
| pd.read_sql(query, conn) | Load rows from a database |
Select / Filter
| df['col'] / df.col | Single column as Series |
| df.loc[row, col] | Label-based selection |
| df.iloc[row, col] | Position-based selection |
| df[df['col'] > x] | Boolean mask filter |
| df['col'].isin([...]) | Membership filter |
| df.query('expr') | SQL-like filter string |
Clean
| df.isna() / df.notna() | Detect missing values |
| df.dropna() | Drop rows/cols with NaN |
| df.fillna(value) | Fill missing values |
| df.drop_duplicates() | Remove duplicate rows |
| df.rename(columns={...}) | Rename labels |
| df.astype(dtype) | Cast column type |
Reshape / Combine
| df.sort_values('col') | Sort rows by value |
| df.groupby('col').agg(...) | Split-apply-combine |
| pd.merge(a, b, on='key') | SQL-style join |
| pd.concat([a, b]) | Stack rows or columns |
| df.pivot_table(...) | Wide summary table |
| df.melt(...) | Wide → long format |
Text / Time / Math
| df['c'].str.lower() | Vectorized string ops |
| pd.to_datetime(col) | Parse dates |
| df['c'].dt.month | Extract date part |
| df.apply(func, axis=1) | Row/column-wise function |
| df['c'].value_counts() | Frequency of values |
| df.corr() | Correlation matrix |
Important points (IMP)
loc includes the end of a slice; iloc excludes it — this is the single most common source of bugs.
Most pandas methods return a NEW object by default (they don't mutate). Reassign the result: df = df.dropna(), or pass inplace=True.
A slice/filter of a DataFrame can be a *view* or a *copy* — pandas will warn 'SettingWithCopyWarning' if you write into an ambiguous one. Use .copy() when you intend to keep working on a filtered subset.
NaN is a float — an integer column with any missing values gets silently upcast to float64.
Always check df.dtypes after read_csv; numbers stored as text, or dates read as plain strings, are the #1 cause of downstream errors.
Vectorize before you loop: df['a'] + df['b'] beats df.apply(...) beats a Python for-loop, often by 10-100x.
Keep these notes offline
Every topic, example, and the cheat sheet above, bundled into a single Markdown file you can keep, print, or drop into Notion / Obsidian.