Measurement · wrangling · inference · modeling · evaluation · communication

Data Science

Data science turns recorded observations into defensible descriptions, inferences, predictions, decisions, and communication. The work begins before modeling, with measurement and provenance, and continues after modeling, with evaluation, interpretation, monitoring, and the consequences of use.

Primary navigation · data practice map

The model is one station in a longer chain from measurement to evidence to action.

These planned branches separate major kinds of work so navigation is clear. Real projects move between them repeatedly: exploration can expose a collection problem, evaluation can force a feature redesign, and communication can reveal that the original metric was answering the wrong question.

Parent fieldFormal Science

DATAData Collection & WranglingHow was the dataset measured, sampled, joined, cleaned, transformed, and documented before analysis began?planned

EDAExploratory Data AnalysisWhich distributions, relationships, groups, anomalies, missing patterns, and scale choices deserve closer attention?planned

STATStatistical Modeling & InferenceWhich probability model represents uncertainty, and what can the sample support about a larger process or population?planned

MLMachine LearningWhich representation, objective, model family, training procedure, and validation design support useful generalization?planned

Question stack

Different analytical goals can use the same dataset while requiring different evidence.

01MeasureWhat does each row, field, label, and missing value actually represent?

02DescribeWhat patterns exist in this dataset before a predictive model is involved?

03InferWhich claims extend beyond the observed sample, and how uncertain are they?

04PredictHow well will a model perform on relevant data it did not train on?

05Explain causeWhat would happen under an intervention, not merely what tends to co-occur?

06CommunicateWhat does the audience need to see to inspect the evidence and its limits?

PIPEData EngineeringHow should data be stored, moved, validated, recomputed, monitored, and served reliably at the required scale?planned

VISVisualization & CommunicationWhich visual encodings reveal the important pattern without hiding uncertainty, scale, missingness, or comparison context?planned

CAUSEExperiments & Causal InferenceWhat intervention or design would distinguish a causal effect from association, confounding, selection, or common causes?planned

EVALResponsible Data Science & EvaluationDoes the measurement remain valid, the evaluation remain independent, and the system remain useful across groups, shifts, and real deployment?planned

Unsupervised-learning instrument

Watch a fitting objective create a partition without turning that partition into ground truth.

deterministic teaching sample

K-means laboratory · unsupervised partitioning

How does alternating assignment and recentering create a partition from unlabeled coordinates?

Choose k, assign each point to its nearest centroid, then move every centroid to the mean of its assigned points. Repeating those two steps reduces the within-cluster squared-distance objective until the assignments stabilize at a local solution.

Model boundary

K-means does not discover a uniquely true set of categories. Results depend on k, initialization, feature scaling, outliers, and the geometry of the data. It works best when squared Euclidean distance and roughly compact clusters are meaningful for the problem.

x₁ → · x₂ ↑

Clusters requested

k = 3chosen before fitting

Completed iterations

0assign + recenter cycles

Within-cluster SSE

unassignedlower for this fixed k is the fitting objective

Choose k

What to watch

With k = 3, the toy sample contains three compact spatial clouds, so the chosen geometry is favorable to K-means. Try k = 2 or k = 4 to see that the algorithm will still produce a partition even when that partition does not match the sample's generating structure.

Evaluation discipline · reference, not navigation

A clean metric cannot rescue a contaminated question.

The most consequential failures often happen outside the optimization loop: a target that does not measure the intended concept, leakage from the future, an evaluation set tuned into a training set, a deployment population that differs from development, or a metric that ignores the cost of mistakes.

01ProvenanceRecord where data came from, how it was measured, when it was collected, and which transformations produced the analytical table.

02LeakageKeep information from the evaluation target or future outside the features available when the prediction would actually be made.

03BaselineCompare a complicated model against simple rules, naive predictions, or established procedures before calling complexity an improvement.

04Held-out evidenceUse genuinely separate data or designs to estimate performance rather than repeatedly tuning against the same evidence.

05ShiftAsk whether the people, environment, measurement process, incentives, or data distribution can change after development.

06Decision costA statistically strong metric can still support a poor system when false positives, false negatives, delay, fairness, privacy, or action costs matter differently.

Three claims that should never be silently merged

Description

What patterns are present in the observed data?

Prediction

How accurately can an outcome be estimated for relevant unseen cases?

Causation

How would the outcome change under an intervention?