Day 1 — What machine learning actually is, and your first model

Your phone unlocked when it saw your face this morning. Your inbox had already moved two emails to spam. At standup someone said "we should add AI to this", and you nodded, because everyone nodded. Then last week an interviewer leaned back and asked, "So — how does a model actually learn?" and the honest answer was that you had never looked inside one.

Today you and I look inside one. By the end of this lesson you'll have a working classifier you wrote yourself, in about forty lines, with no library — and you'll be able to say what "learning" means without waving your hands.

This is Day 1, and it's free. If it clicks, the next fourteen days build the rest by hand: the data pipeline, gradient descent, logistic regression and the metrics that matter, trees, clustering, a neural network from scratch, attention and transformers, LLMs and RAG, ML system design, and a timed mock round on Day 15. Every day ends with an assignment; every next morning opens with its solution.

Let's start at a fruit stall

You're picking mangoes. You squeeze one: it gives a little. Ripe or not? You don't run a formula. You compare it to every mango you've ever squeezed — the ones that gave like this were mostly ripe, so you say ripe. You bite. Wrong — it was sour. Next time a mango gives exactly like that, you'll lean the other way.

That's the whole subject in one paragraph. Two things happened:

1. You judged by similarity to what you'd seen before. That is the model you'll build today — k-nearest neighbours, or kNN. Find the most similar past cases, let them vote.
2. You got better by being told when you were wrong. That is the learning loop, and every model in this course — from today's kNN to Day 12's transformer — is a variation of it.

You met the same two moves before breakfast. Your spam folder is a vote among emails that looked like this one. "Customers also bought" is a vote among shoppers whose baskets looked like yours. Face unlock measures the distance between the face in front of the camera and the one you enrolled, and lets you in when the distance is small enough. Similarity, then a decision — with a nudge every time someone hits "not spam".

What it actually looks like

Before any code, here is the loop. Keep it in view for the whole course, because every day will just swap what sits in the boxes.

Data is rows you've seen, with the answer attached. The model is a function with adjustable parts that turns a row into a guess. The loss is a number saying how wrong the guess was. The update nudges the adjustable parts so the loss drops next lap. Go round enough times and the guesses get good — that's learning.

kNN is the one model where the loop is almost degenerate: its adjustable parts are the stored rows, so "training" is just remembering, and the nudge is "add the mango you just bit". That makes it the perfect first model — you get to see similarity and voting cleanly, before Day 3 adds real knobs and real updates.

Step 1 — Put the words on a map

Every ML interview opens with vocabulary, and the words nest.

Artificial intelligence is the widest circle — anything that makes a machine act smart, including a chess engine whose every move is a rule a human typed. Machine learning is the part where the behaviour is learned from data rather than written by hand. Deep learning is machine learning where the model is a stack of layers (a neural network). Generative AI is deep learning whose output is new content — text, images, code. An LLM is a generative deep-learning model; today's kNN is machine learning that never touches a layer.

Step 2 — The three ways to learn

The loop has three flavours, and the interviewer wants you to name them from an example, not from a definition.

Supervised learning: every row comes with the answer. Squeeze, plus "was it ripe" — you learn a map from the measurements to the label. Spam or not, ripe or not, price of a house: nearly everything you'll build in the first week.

Unsupervised learning: rows with no answer column. You have a thousand baskets and no one told you what the "groups" are; the model finds them. "Customers also bought" starts here (Day 8).

Reinforcement learning: there are no rows at all until you act. An agent takes an action, the world hands back a reward, and the update pushes toward actions that earn more. Game-playing bots and the fine-tuning stage of chat models live here — it's a name-drop today, not a build.

The question that sorts them: do you have the answer column? Yes → supervised. No → unsupervised. "There isn't a table, only a game" → reinforcement.

Step 3 — Data is a table: features and a label

Supervised learning starts with a table, and the interviewer will expect you to name its parts without pausing.

Each row is one thing you observed — one mango. The columns you measured are the features, written x (give when squeezed, sugar content). The column you want to predict is the label, written y (ripe = 1, not ripe = 0). A model is a function from the x columns to the y column. That sentence is the whole job.

You'll need fruit to squeeze, so here's a generator with a known truth baked in: ripe fruit gives more and is sweeter, with plenty of noise so the classes overlap. Same seed, same fruit, every run.

package dev.fiveyear.ml.knn;
 
import java.util.Random;
 
/** Synthetic fruit: two features (squeeze give 0–10, sugar in °Bx) and a label (1 = ripe). */
public final class FruitData {
    public final double[][] x;
    public final int[] y;
 
    private FruitData(double[][] x, int[] y) {
        this.x = x;
        this.y = y;
    }
 
    public static FruitData generate(int n, long seed) {
        Random rng = new Random(seed);
        double[][] x = new double[n][2];
        int[] y = new int[n];
        for (int i = 0; i < n; i++) {
            boolean ripe = i % 2 == 0;                      // alternate so classes stay balanced
            y[i] = ripe ? 1 : 0;
            x[i][0] = (ripe ? 6.5 : 3.5) + 1.2 * rng.nextGaussian();   // ripe fruit gives more
            x[i][1] = (ripe ? 14 : 10) + 2.0 * rng.nextGaussian();     // and is sweeter
        }
        return new FruitData(x, y);
    }
}

The first six rows, with the seed above, look like (7.9, 15.8) → ripe, (2.4, 7.8) → not, (6.8, 15.4) → ripe, (2.5, 7.2) → not, (6.3, 17.0) → ripe, (4.5, 9.8) → not. Numbers with a label attached — that's a dataset.

Step 4 — Distance is similarity

"Compare it to every mango you've squeezed" needs a number for how alike two rows are. The obvious one is the straight-line distance between them, treating each row as a point:

Two mangoes with the same give and the same sugar sit on top of each other — distance zero, perfectly similar. That's the first piece of the course's tiny core library, Vec: four operations you'll reuse every single day.

package dev.fiveyear.ml.core;
 
/** The four vector operations the whole course is built on. */
public final class Vec {
    private Vec() {}
 
    public static double dot(double[] a, double[] b) {
        double sum = 0;
        for (int i = 0; i < a.length; i++) sum += a[i] * b[i];
        return sum;
    }
 
    /** Euclidean distance — the straight-line gap between two rows. */
    public static double dist(double[] a, double[] b) {
        double sum = 0;
        for (int i = 0; i < a.length; i++) {
            double d = a[i] - b[i];
            sum += d * d;
        }
        return Math.sqrt(sum);
    }
 
    public static double[] add(double[] a, double[] b) {
        double[] out = new double[a.length];
        for (int i = 0; i < a.length; i++) out[i] = a[i] + b[i];
        return out;
    }
 
    public static double[] scale(double[] a, double k) {
        double[] out = new double[a.length];
        for (int i = 0; i < a.length; i++) out[i] = a[i] * k;
        return out;
    }
}

The failure: whichever column is biggest wins

Now watch distance lie to you. Measure fruit in grams and in "give" (a 0–10 scale) and ask which of two neighbours is closer to a 150 g mango that gives 3.0:

package dev.fiveyear.ml.knn;
 
import dev.fiveyear.ml.core.Scaler;
import dev.fiveyear.ml.core.Vec;
 
public class UnitsTrap {
    public static void main(String[] args) {
        // [weight in grams, squeeze give 0–10] — the same fruit, two very different rulers
        double[] query = {150, 3.0};
        double[] firmSibling = {160, 9.0};   // 10 g heavier, but rock hard
        double[] softSibling = {130, 3.2};   // 20 g lighter, gives exactly like the query
        System.out.printf("raw:    firm %.1f  soft %.1f%n",
                Vec.dist(query, firmSibling), Vec.dist(query, softSibling));
 
        double[][] all = {query, firmSibling, softSibling};
        double[][] z = new Scaler().fit(all).transform(all);
        System.out.printf("scaled: firm %.2f  soft %.2f%n",
                Vec.dist(z[0], z[1]), Vec.dist(z[0], z[2]));
    }
}

raw:    firm 11.7  soft 20.0
scaled: firm 2.30  soft 1.61

Raw, the rock-hard mango is "nearer" — a 20 g difference in weight outweighs the entire 0–10 give scale, so the feature that actually predicts ripeness barely gets a vote. The distance formula doesn't know grams from give; it only sees numbers, and the biggest numbers dominate.

The fix is to put every column on the same ruler: subtract the column's mean, divide by its standard deviation, so every feature is measured in "how many standard deviations from typical":

package dev.fiveyear.ml.core;
 
/** Standardise every column to mean 0, std 1 — so no unit gets to shout. */
public final class Scaler {
    public double[] mean, std;
 
    public Scaler fit(double[][] x) {
        int d = x[0].length;
        mean = new double[d];
        std = new double[d];
        for (double[] row : x)
            for (int j = 0; j < d; j++) mean[j] += row[j] / x.length;
        for (double[] row : x)
            for (int j = 0; j < d; j++) std[j] += (row[j] - mean[j]) * (row[j] - mean[j]) / x.length;
        for (int j = 0; j < d; j++) std[j] = std[j] == 0 ? 1 : Math.sqrt(std[j]);
        return this;
    }
 
    public double[][] transform(double[][] x) {
        double[][] out = new double[x.length][mean.length];
        for (int i = 0; i < x.length; i++)
            for (int j = 0; j < mean.length; j++) out[i][j] = (x[i][j] - mean[j]) / std[j];
        return out;
    }
}

After scaling, the mango that gives the same is the nearest, which is the answer a human would have given. Tonight's assignment measures exactly how much accuracy this trap costs on a full dataset.

Step 5 — Predict by letting the neighbours vote

With distance in hand the algorithm is two lines of English: sort the stored rows by distance to the new one, take the closest k, and return the label most of them carry.

Here's the query from the diagram — give 5.0, sugar 12.0, right in the overlap. Four of its five nearest fruit are ripe, so the vote says ripe.

    /** Indices of the k rows closest to the query, nearest first. */
    public int[] neighbours(double[] query) {
        Integer[] idx = new Integer[x.length];
        for (int i = 0; i < idx.length; i++) idx[i] = i;
        Arrays.sort(idx, Comparator.comparingDouble(i -> Vec.dist(x[i], query)));
        int[] out = new int[Math.min(k, idx.length)];
        for (int i = 0; i < out.length; i++) out[i] = idx[i];
        return out;
    }
 
    /** Majority vote among the neighbours; a tie goes to the nearest of the tied labels. */
    public int predict(double[] query) {
        int[] votes = new int[Arrays.stream(y).max().orElse(0) + 1];
        int best = -1;
        for (int i : neighbours(query)) {
            votes[y[i]]++;
            if (best < 0 || votes[y[i]] > votes[best]) best = y[i];
        }
        return best;
    }

Two details carry weight in interviews. First, fit stores the rows and does nothing else — there is no training cost, which is why kNN is called a lazy learner; every millisecond is paid at prediction time, scanning all n rows for each query. Second, k is odd in every example here so a two-class vote can't tie; the code still breaks a tie toward the nearest of the tied labels because a production caller will pass an even k eventually.

Step 6 — Choosing k, and the first look at overfitting

k is not learned from the data — you choose it. Choose badly in either direction and the same code gives a worse model:

package dev.fiveyear.ml.knn;
 
public class ChooseK {
    public static void main(String[] args) {
        FruitData train = FruitData.generate(30, 42);
        FruitData test = FruitData.generate(15, 7);
        for (int k : new int[] {1, 5, 15, 29}) {
            double acc = new Knn(k).fit(train.x, train.y).score(test.x, test.y);
            System.out.printf("k=%2d  accuracy=%.2f%n", k, acc);
        }
        // k= 1  accuracy=0.80   <- copies every quirk of the single nearest fruit
        // k= 5  accuracy=0.93
        // k=15  accuracy=0.93
        // k=29  accuracy=0.80   <- 29 of 30 rows vote: the majority blurs the edge away
    }
}

At k = 1 the model copies the single nearest fruit, quirks and all — every odd mango in the training set gets its own little island of wrong answers. At k = 29, twenty-nine of the thirty rows vote on every query, so whichever class has one more member wins everywhere and the boundary blurs away. In between, the vote averages out the odd fruit while still following the real edge.

You've just met the central tension of the whole course. A model that follows the training data too closely memorises (it overfits); a model too blunt to follow the real pattern underfits. Day 5 gives it a name — bias versus variance — and a proper toolkit. For now, one rule: pick k on rows the model hasn't been scored on, never on the test set, and tonight's assignment gives you the leave-one-out trick to do that with only thirty fruit.

kNN vs a rule you write by hand

The obvious alternative to any first model is an if statement: "ripe if give is above 5 and sugar above 12". Here's why you'd pick one over the other.

The row that decides real projects is the second one. kNN with ten million rows pays ten million distance computations per prediction. That cost is what pushes you toward models with a handful of knobs — which is Day 3.

The complete implementation

Everything assembled: the core Vec above, the classifier, and a demo whose comments state the exact output you'll see.

package dev.fiveyear.ml.knn;
 
import dev.fiveyear.ml.core.Vec;
import java.util.Arrays;
import java.util.Comparator;
 
/** k-nearest neighbours: remember everything, predict by asking the k closest rows to vote. */
public final class Knn {
    private final int k;
    private double[][] x;
    private int[] y;
 
    public Knn(int k) {
        this.k = k;
    }
 
    /** "Training" is just remembering — there is nothing to fit. */
    public Knn fit(double[][] x, int[] y) {
        this.x = x;
        this.y = y;
        return this;
    }
 
    /** Indices of the k rows closest to the query, nearest first. */
    public int[] neighbours(double[] query) {
        Integer[] idx = new Integer[x.length];
        for (int i = 0; i < idx.length; i++) idx[i] = i;
        Arrays.sort(idx, Comparator.comparingDouble(i -> Vec.dist(x[i], query)));
        int[] out = new int[Math.min(k, idx.length)];
        for (int i = 0; i < out.length; i++) out[i] = idx[i];
        return out;
    }
 
    /** Majority vote among the neighbours; a tie goes to the nearest of the tied labels. */
    public int predict(double[] query) {
        int[] votes = new int[Arrays.stream(y).max().orElse(0) + 1];
        int best = -1;
        for (int i : neighbours(query)) {
            votes[y[i]]++;
            if (best < 0 || votes[y[i]] > votes[best]) best = y[i];
        }
        return best;
    }
 
    public int[] predict(double[][] queries) {
        int[] out = new int[queries.length];
        for (int i = 0; i < queries.length; i++) out[i] = predict(queries[i]);
        return out;
    }
 
    /** Fraction of queries whose vote matched the true label. */
    public double score(double[][] queries, int[] truth) {
        int[] pred = predict(queries);
        int hits = 0;
        for (int i = 0; i < truth.length; i++) if (pred[i] == truth[i]) hits++;
        return (double) hits / truth.length;
    }
}

package dev.fiveyear.ml.knn;
 
public class Demo {
    public static void main(String[] args) {
        FruitData train = FruitData.generate(30, 42);  // the 30 fruit you have squeezed
        FruitData test = FruitData.generate(15, 7);    // 15 new ones — a different seed,
                                                       // or you'd be testing on your own notes
        Knn knn = new Knn(5).fit(train.x, train.y);
 
        int guess = knn.predict(new double[] {5.0, 12.0});
        System.out.println("give 5.0, sugar 12.0 -> " + guess);   // give 5.0, sugar 12.0 -> 1
 
        System.out.printf("accuracy %.2f%n", knn.score(test.x, test.y)); // accuracy 0.93
    }
}

Fourteen of fifteen new fruit sorted correctly, from thirty stored rows and no formula for ripeness anywhere in the code. The one it missed sits deep in the overlap where the ripe and unripe clouds cross — every model will miss some of those, and Day 4 teaches you which kind of miss you should mind more.

The questions asked most

"Supervised or unsupervised — how do you tell?" Look for the answer column. Labelled rows → supervised (classification if the label is a category, regression if it's a number). No labels → unsupervised: the model finds structure such as clusters. Reinforcement learning is the odd one out: no table at all, only actions and rewards.

"What's a hyperparameter?" A setting you choose before training rather than one the data sets: k here, the learning rate on Day 3, the tree depth on Day 6. Parameters are the values the loop adjusts. kNN is unusual in having a hyperparameter and no parameters at all.

"Why is kNN called a lazy learner, and why does that matter?" It does no work at fit time and all of it at query time — O(n · d) per prediction. For a handful of rows that's free; for millions it's the bottleneck. The remedies are a spatial index (a k-d tree or ball tree in low dimensions, approximate nearest-neighbour search like HNSW in high ones) or a model that compresses the data into a few numbers, which is where linear and logistic regression come in.

"How do you choose k?" Odd, so votes don't tie; small enough to follow the real boundary, large enough to average out noise; chosen by validation — leave-one-out or k-fold on the training rows — and never by peeking at the test set. A square-root-of-n starting point is fine to mention as a heuristic, not a rule.

"Why does kNN struggle with many features?" In high dimensions the distances between points bunch together — the nearest neighbour isn't much nearer than the farthest — so "closest" stops meaning "most similar". Fewer, better features or a learned projection (Day 8's PCA) is the fix.

Assignment — Day 1: kNN, properly

Goal — Turn today's classifier into one you'd trust, by measuring the two things that silently break it: unscaled features and an unvalidated k.

Tasks

1. Scaling, measured. Write a generator with the same shape as FruitData but a second feature that is pure noise and a hundred times bigger: per row (ripe rows first, alternating), give = (ripe ? 7 : 3) + 1.0 * g then weight = 180 + 60 * g grams, each g a fresh nextGaussian(). Take 40 training rows with seed 42 and 20 test rows with seed 7. Print the k = 5 accuracy on the raw features, then again after a Scaler fitted on the training rows only and applied to both.
2. Leave-one-out. On FruitData.generate(30, 42), for each k in 1, 3, 5, …, 15: hold one row out, fit Knn on the other 29, predict the held row, repeat for all 30 and count the hits. Print the accuracy per k and the best k.
3. Weighted kNN. Give each neighbour a vote of 1 / distance instead of 1 (add a tiny epsilon so a zero distance can't divide by zero). Compare it with the plain vote at k = 5 on today's train/test split.

Acceptance checks

• Task 1: raw accuracy prints 0.40 (must be at or below 0.70); scaled prints 0.95 (must be at or above 0.90).
• Task 2: leave-one-out prints k=1 0.933, k=3 0.833, k=5 0.967, k=7 0.967, k=9 0.900, k=11 0.900, k=13 0.867, k=15 0.833; best k = 5 (first of the tied maxima) at 0.967.
• Task 3: plain 0.93, weighted 0.87. Yes, lower — with fifteen test rows one flipped vote costs seven points. Write one comment explaining why weighting can hurt when a query's single nearest neighbour is an odd fruit.

Stretch — A KnnRegressor that averages the neighbours' numeric targets; on a set where sugar rises with give, it should beat predicting the mean (ours: MSE 1.08 against 4.08).

Interview tie-in — "Why does feature scaling matter for kNN but not for a decision tree?" After Task 1 you answer from a number you printed: distance adds columns together, so the biggest unit dominates; a tree (Day 6) asks one column at a time — "is give above 5?" — and never adds grams to give.

The interview corner

Clarifying questions to ask first

• Is there a label column, and is it a category or a number? (Supervised or not; classification or regression.)
• Which mistake is more expensive — a ripe mango called unripe, or the reverse? (Decides the metric long before it decides the model.)
• How many rows, how many features, and are the features on comparable scales? (Decides whether kNN is even on the table.)

The follow-up ladder

1. "Ten million training rows and a latency budget of 5 ms — kNN scans them all. Now what?" — A spatial index for low dimensions (k-d tree), approximate nearest neighbours for high ones (HNSW, the engine behind vector databases), or switch to a parametric model that compresses the rows into a few weights.
2. "Only 2% of the fruit are ripe." — A majority vote drowns the rare class and accuracy reads 0.98 for a model that never says "ripe". Weight the votes, resample the training rows, and report precision and recall instead (Day 4).
3. "One feature is the variety — mango, banana, papaya." — Distance between category names is undefined. One-hot encode it (Day 2), and consider whether Euclidean distance still makes sense with mixed columns.
4. "The supplier changed and the model has got worse every month." — Data drift. The model isn't wrong; the world moved. Monitor accuracy on fresh labelled samples and schedule retraining (Day 14 builds the monitoring).
5. "Predict the sugar content, not ripe/unripe." — Regression: average the neighbours' targets instead of voting (the stretch task). Then note that a fitted line does the same job with two numbers instead of thirty rows — Day 3.

Mistakes that fail the round

• Choosing k by trying values against the test set. The moment you do, the test score stops being an estimate of anything.
• Forgetting to scale, or fitting the scaler on all the rows including the test ones. The first gives a bad model; the second gives a good-looking lie.
• Calling 0.93 on fifteen rows a result. One row is seven points; say "about 90%, and I'd want a larger held-out set before I trusted the decimal".

Where to go from here

Everything you built today has a one-line equivalent in Python's scikit-learn (KNeighborsClassifier), and when you use it you'll now know what n_neighbors does, why the docs nag you to scale, and why it gets slow. The reading path below covers the three data-structure ideas kNN leans on: the hash map behind "have I seen this row", the heap that makes "top-k nearest" cheap, and the Big-O that decides whether kNN survives contact with production.

Tomorrow, Day 2 is about the part of the job nobody puts on the slide: the data. You'll build the Matrix, Dataset and Split utilities the next thirteen days run on, and reproduce the leak the callout above warned about — with tonight's kNN reporting a number that's flatly too good.