Machine Learning

Machine Learning (ML) is a branch of Artificial Intelligence where a computer program improves at a task by learning patterns from data, instead of following rules that a human explicitly wrote. Rather than coding "if this, then that," we show the algorithm many examples and let it work out the underlying relationship on its own.

Formally, Tom Mitchell's definition is often quoted: a program learns from experience E with respect to task T and performance measure P, if its performance at T, measured by P, improves with experience E. In practice this means: more (good) data and more training generally produce a better model.

Supervised Learning

The model learns from labeled data — each training example has an input and a known correct output. The goal is to learn a mapping from input to output so it can predict labels for new, unseen inputs.

• Regression: predicting a continuous number (e.g. house price, temperature).
• Classification: predicting a category (e.g. spam vs. not spam, disease vs. healthy).

Unsupervised Learning

The model works with unlabeled data and tries to find hidden structure on its own — grouping similar items together (clustering) or reducing the number of variables while keeping the important signal (dimensionality reduction).

Reinforcement Learning

An agent interacts with an environment, takes actions, and receives rewards or penalties. Over many trials it learns a policy that maximizes cumulative reward — the approach behind game‑playing AI (e.g. AlphaGo) and robotics.

Features & Labels

A feature is an input variable (e.g. square footage, age, pixel value). A label is the answer we want to predict (e.g. house price). A dataset is usually organized as a table where rows are examples and columns are features, with one column reserved as the label.

Train / Validation / Test Split

Data is split so we can measure how well a model generalizes to data it has never seen:

• Training set (~70%): used to fit the model's parameters.
• Validation set (~15%): used to tune hyperparameters and pick the best model.
• Test set (~15%): used once, at the end, to report unbiased final performance.

Overfitting, Underfitting & the Bias‑Variance Tradeoff

Underfitting happens when a model is too simple to capture the pattern in the data (high bias). Overfitting happens when a model memorizes the training data, including its noise, and fails to generalize (high variance). The goal is the balanced middle ground.

Linear Regression

Fits a straight line (or hyperplane) y = w·x + b through the data, minimizing the mean squared error between predictions and actual values.

from sklearn.linear_model import LinearRegression
import numpy as np

X = np.array([[650], [800], [1200], [1500], [2000]])  # sq. ft.
y = np.array([70000, 90000, 140000, 175000, 240000])    # price

model = LinearRegression()
model.fit(X, y)

print("Slope (price per sq ft):", model.coef_[0])
print("Predicted price for 1000 sq ft:", model.predict([[1000]])[0])

Logistic Regression

Despite the name, it's a classification algorithm. It squashes a linear combination of features through a sigmoid function to output a probability between 0 and 1.

from sklearn.linear_model import LogisticRegression

# X: [hours_studied], y: 0 = fail, 1 = pass
model = LogisticRegression()
model.fit(X_train, y_train)

probability_of_passing = model.predict_proba([[5]])[0][1]
print(f"Predicted pass probability: {probability_of_passing:.2f}")

Decision Trees & Random Forests

A decision tree splits the data repeatedly on feature thresholds (e.g. "is age > 30?") to form a tree of decisions. A random forest trains many trees on random subsets of data/features and averages their votes — this reduces overfitting and usually improves accuracy.

K‑Nearest Neighbors (KNN)

To classify a new point, KNN looks at the k closest labeled points in the training set and takes a majority vote. Simple, but can be slow on large datasets since it compares against every stored example.

K‑Means Clustering

An unsupervised algorithm that groups data into k clusters by repeatedly assigning points to the nearest cluster center, then recomputing each center as the mean of its assigned points.

from sklearn.cluster import KMeans

# Customer data: [annual_spend, visits_per_month]
kmeans = KMeans(n_clusters=3, random_state=42, n_init="auto")
kmeans.fit(customer_data)

print("Cluster for each customer:", kmeans.labels_)
print("Cluster centers:", kmeans.cluster_centers_)

Neural Networks

Inspired loosely by the brain, a neural network is made of layers of connected "neurons." Each connection has a weight; each neuron applies an activation function. Stacking many layers ("deep learning") lets the network learn very complex patterns, powering image recognition, translation, and large language models.

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(16, activation="relu", input_shape=(3,)),
    tf.keras.layers.Dense(8, activation="relu"),
    tf.keras.layers.Dense(1, activation="sigmoid"),
])

model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(X_train, y_train, epochs=20, validation_split=0.2)

Choosing the right metric matters as much as choosing the right algorithm.

For Regression

• MAE (Mean Absolute Error): average absolute difference between predicted and actual values.
• MSE / RMSE: penalizes larger errors more heavily by squaring them.
• R² score: proportion of variance in the target explained by the model (closer to 1 is better).

For Classification

• Accuracy: % of correct predictions overall.
• Precision: of everything predicted positive, how much was actually positive.
• Recall: of everything actually positive, how much did we catch.
• F1 score: harmonic mean of precision and recall — useful when classes are imbalanced.

from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score, confusion_matrix
)

y_pred = model.predict(X_test)

print("Accuracy :", accuracy_score(y_test, y_pred))
print("Precision:", precision_score(y_test, y_pred))
print("Recall   :", recall_score(y_test, y_pred))
print("F1 score :", f1_score(y_test, y_pred))
print("Confusion matrix:\n", confusion_matrix(y_test, y_pred))

Example 1 — Predicting House Prices (Regression)

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error

df = pd.read_csv("houses.csv")           # columns: sqft, bedrooms, age, price
X = df[["sqft", "bedrooms", "age"]]
y = df["price"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

model = LinearRegression()
model.fit(X_train, y_train)

predictions = model.predict(X_test)
print("MAE:", mean_absolute_error(y_test, predictions))

Example 2 — Classifying Iris Flowers (Classification)

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

data = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    data.data, data.target, test_size=0.25, random_state=1
)

clf = RandomForestClassifier(n_estimators=100, random_state=1)
clf.fit(X_train, y_train)

print("Test accuracy:", accuracy_score(y_test, clf.predict(X_test)))

Example 3 — Customer Segmentation (Clustering)

A retailer wants to group customers by spending habits to target marketing campaigns. With no labels available, K‑Means groups customers into, say, three clusters: budget shoppers, regular shoppers, and big spenders — letting the business tailor offers to each group.

Real ML projects follow a repeatable pipeline, rarely a straight line — you often loop back a step when results aren't good enough.

1. Collect data: gather examples relevant to the problem.
2. Preprocess: clean missing values, encode categories, scale numbers.
3. Train: fit one or more algorithms on the training set.
4. Evaluate: measure performance on validation/test data, tune hyperparameters.
5. Deploy: serve the model behind an API or embed it in an app, then monitor it over time.

Machine Learning is about learning patterns from data rather than hand-coding rules. Start with the three core paradigms (supervised, unsupervised, reinforcement), get comfortable with the train/validation/test workflow and the bias‑variance tradeoff, then practice with a handful of classic algorithms — linear/logistic regression, trees, KNN, K‑Means, and a basic neural network — on small, real datasets.