Introduction

Machine learning is the practice of building systems that improve at a task by learning patterns from data rather than following hand-written rules. Instead of specifying exactly how to classify an email or predict a price, we define an objective, provide examples, and let an algorithm find parameters that perform well—and, crucially, that generalize to data it has not seen.

This section covers the classical methods that remain foundational and widely used. They are often faster, more interpretable, and more data-efficient than deep networks, and they teach the concepts—generalization, bias and variance, evaluation—that apply to every model.

Learning Paradigms

Most problems fall into a few paradigms. In supervised learning, models learn from labeled examples to predict an output, as in classification and regression. In unsupervised learning, models find structure in unlabeled data through clustering, dimensionality reduction, and density estimation. Reinforcement learning frames learning as an agent taking actions to maximize reward over time. Many real systems combine these, for example using unsupervised pretraining followed by supervised fine-tuning.

Classical Algorithms

A toolbox of well-understood algorithms covers a large share of practical problems: linear and logistic regression, decision trees and ensembles such as random forests and gradient boosting, support vector machines, k-nearest neighbors, and naive Bayes. Each carries assumptions and trade-offs in accuracy, speed, and interpretability. Choosing well depends on the data's size and shape, the cost of errors, and the need to explain predictions.

Evaluation and Generalization

A model is only as trustworthy as its evaluation. Splitting data into training, validation, and test sets—or using cross-validation—estimates how a model behaves on unseen data. Metrics must match the goal: accuracy can mislead on imbalanced data, where precision, recall, and calibration tell a fuller story. The central tension is overfitting versus underfitting: regularization, more data, and simpler models curb memorization, while richer models and features address underfitting.

The Workflow

Building a model is iterative. It begins with framing the problem and assembling and cleaning data, followed by feature engineering, model selection, and tuning. Disciplined experiment tracking and honest evaluation prevent subtle leakage and inflated results. The habits formed here—careful data handling and skeptical measurement—carry directly into deep learning and production systems.