Neural Network Fundamentals and Backpropagation

A neural network is not a new idea bolted onto everything you have learned so far. It is the same building blocks stacked and connected: a linear combination of inputs, a nonlinear squashing function, and gradient descent to fit the parameters. A single sigmoid neuron is exactly logistic regression. Put a few of those neurons in a layer, stack the layers, and you have a network that can represent functions a single linear model never could. This lesson builds the network up from that one neuron, shows the forward pass that turns inputs into a prediction, and then derives backpropagation, the algorithm that computes every gradient efficiently so gradient descent can train the whole thing. Backpropagation is one of the most reliable conceptual questions in a quant interview, so we work a full forward and backward pass by hand.