The 15-Day AI/ML Interview Bootcamp
Day 10 — Training deep networks: initialisation, Adam, dropout, and early stopping
Why deep networks stall — vanishing gradients — and the toolkit that fixes it: Xavier and He initialisation, momentum and Adam, batch normalisation, dropout, learning-rate schedules and early stopping, built from scratch — with the machine learning interview questions on training dynamics.
You finished Day 9 with a network that solves XOR and untangles a spiral, so you did the obvious next thing: made it deeper. Six layers instead of two. You hit run, and the loss printed 0.700, then 0.695, then 0.694, and then 0.694 for two hundred epochs. Nothing crashed. The gradient check passes. Every line of code is the same code that worked yesterday. The network is simply refusing to learn, and the number it refuses at — 0.693 — is ln 2: the cross-entropy of shrugging and guessing 50/50 on every row. That stall is the most common thing a deep-learning interviewer wants you to explain, be…
In this lesson
Unlock the rest of The 15-Day AI/ML Interview Bootcamp
Day 10 — Training deep networks: initialisation, Adam, dropout, and early stopping is part of the full course. One payment unlocks every lesson, every other course, and every premium article — for life.