The Inner Product and Orthogonality

Up to now, we have added and multiplied vectors and matrices with each other. There is much more to be said about vectors. Take for example a vector a and b. We might be interested in the length of a vector, the distance between vectors or the angle between vectors. Analysing these properties might help a lot for making inferences on certain topics, which will be discussed later on this course.

Geometry of two vectors.
Figure 1 – Geometry of two vectors.

Analysing these properties might help us to fit a model, by fitting a line through a cloud of data points. In order to fit such a line, we need to know the distances from each point in the cloud up to the line – more specific – the shortest distance from each point in the cloud up to the line. Before we get into these topics, let’s take a look at the mathematics behind the geometrical properties.