Mathematics for AI: Vectors

In the context of AI, a vector is an ordered list of numbers (i.e. the order matters—changing the order changes the meaning). Each number in the vector represents a dimension. For example, this is a 3D vector:

[23, -1, 4]

You can plot the dimensions of a vector on a graph. For example, here’s the vector (23, -1, 4) visualized as an arrow from the origin (0, 0, 0):

Vectors are a structured way to represent data in AI, allowing machines to understand and process complex inputs.

Once data is represented as vectors, AI systems perform various mathematical operations—such as addition, multiplication, etc—to transform data, identify patterns, make predictions and generate insights.

Let’s take a closer look at the key vector operations and see how they work in practice.

Vector Addition

Consider two vectors A and B:

A = [-4, 3]
B = [5, 3]

To add them, you add the corresponding elements to get a new vector. For example:

If
A = [-4, 3] and B = [5, 3]
Then
A + B = [-4 + 5, 3 + 3] = [1, 6]

Note: Vector addition only works if the vectors are the same size.

Vector Multiplication

There are several ways to multiply vectors. We’ll focus on the most common types used in AI:

• Dot product
• Element-wise multiplication
• Scalar multiplication

Dot Product

Calculating the dot product of two vectors gives a scalar result—that is, a single number. Consider the vectors A and B:

A = [2, 6, 4]
B = [5, 3, 10]

To find the dot product, multiply the corresponding elements:

(2)(5) = 10
(6)(3) = 18
(4)(10) = 40

Then, add these products together:

10 + 18 + 40 = 68

So, the dot product is:

A ⋅ B = 68

Element-wise Multiplication

Element-wise multiplication involves multiplying corresponding elements of two vectors of the same length to produce a new vector.

For example, consider the two vectors A and B:

A = [2, 6, 4]
B = [5, 3, 10]

To perform element-wise multiplication, multiply each pair of matching elements:

(2)(5) = 10
(6)(3) = 18
(4)(10) = 40

Then, collect these products into a new vector:

[10, 18, 40]

So, the result of element-wise multiplication is:

A ⊙ B (called the Hadamard product) = [10, 18, 40]

Scalar Multiplication

Scalar multiplication means multiplying every element in a vector by a scalar to produce a new vector. For example, take the vector A:

A = [2, 6, 4]

Now multiply it by the scalar 3:

(2)(3) = 6
(6)(3) = 18
(4)(3) = 12

The resulting vector is:

[6, 18, 12]

3 ⋅ A = [6, 18, 12]

This operation scales the entire vector by the same factor.