01
Data & task
Define examples, inputs, targets or learning signal, sampling process, and the performance question the model is meant to answer.
Data · model · objective · optimization · generalization
Machine Learning
Machine learning fits computational models from experience. A learning system is defined not only by its model architecture, but also by the data-generating process, training objective, optimization method, evaluation design, and the conditions under which performance must generalize.
Learning loop
Training changes parameters; evaluation asks whether the learned rule survives new examples.
A low training error is evidence that optimization found a model that fits the training examples. It is not yet evidence that the model learned the structure we care about.
02
Model
Choose a family of functions or representations capable of mapping inputs to useful predictions, scores, clusters, or features.
03
Objective
Quantify error, likelihood, margin, reconstruction quality, or another signal that expresses what parameter settings should improve.
04
Optimization
Adjust model parameters to improve the training objective using update rules, gradients, closed-form solutions, search, or other algorithms.
05
Evaluation
Measure performance on data not used to fit the model, using metrics and comparisons appropriate to the actual task and decision context.
Perceptron learning lab
Update the weights when the classifier makes a mistake.
A perceptron computes a weighted sum, applies a threshold, compares the prediction with a target, and nudges its parameters when the answer is wrong. The same rule can learn AND and OR, but no single straight decision boundary can represent XOR.
input
(0, 0)
target 0pred 1
input
(0, 1)
target 0pred 1
input
(1, 0)
target 0pred 1
input
(1, 1)
target 1pred 1
w₁
0.0
w₂
0.0
bias
0.0
ŷ = step(0.0x₁ + 0.0x₂ + 0.0)
Learning paradigms · reference
The source of the learning signal changes the problem.
Supervised learning
Examples include desired targets. The model learns a mapping useful for classification, regression, ranking, forecasting, or other predictive tasks.
Unsupervised learning
No explicit target label is supplied. Methods seek structure such as clusters, latent factors, density, or lower-dimensional representations.
Self-supervised learning
Targets are constructed from the data itself, allowing models to learn representations by predicting masked, missing, future, or transformed information.
Generalization guardrails
A model is useful only inside conditions where its evaluation remains informative.
Architecture, data, metrics, and deployment context are coupled. A benchmark score detached from how examples were sampled and how the model will be used is easy to overinterpret.
Training ≠ evaluation
A model can memorize or exploit quirks of its training data. Performance must be measured on appropriately separated validation or test data.
Capacity
A model class must be expressive enough to capture relevant structure but can become flexible enough to fit noise or accidental patterns.
Inductive bias
Architecture, features, regularization, priors, augmentation, and optimization all influence which solutions are easier for a learning system to discover.
Distribution shift
Evaluation is only predictive when future data resembles the conditions represented by the test process. Changing populations or environments can invalidate old metrics.