Vectors

Import

First we shall import the NumPy package.

import numpy as np

Python Lists

Consider the following vectors:

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \mathbf{y} = \begin{bmatrix} 4\\ 5\\ 6 \end{bmatrix} \]

One of the simplest operations that we can think of is to add these two vectors:

\[ \mathbf{z} = \mathbf{x} + \mathbf{y}= \begin{bmatrix} 5\\ 7\\ 9 \end{bmatrix} \]

Let us first see how this is done in native Python, using lists:

x = [1, 2, 3]
y = [4, 5, 6]
z = [ ]
for x_i, y_i in zip(x, y):
    z.append(x_i + y_i)
z

[5, 7, 9]

In a moment, we shall study how NumPy simplifies this operation. But before that, we need to understand how vectors are represented in NumPy.

Vectors as NumPy arrays

Consider the following vector:

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} \]

In NumPy:

x = np.array([1, 2, 3])
x

array([1, 2, 3])

We can also arrive at this using the arange method. The arange method in NumPy is similar to the range function in Python. The right end-point is excluded and the step size is \(1\) by default.

x = np.arange(1, 4)
x

array([1, 2, 3])

Notice that x is a new kind of object and is called a NumPy array. To be more precise, it is an object of type ndarray.

type(x)

numpy.ndarray

We can now turn to vector addition in NumPy.

Vector addition

Addition is one of the elementary operations that can be performed on vectors. Given two vectors

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \mathbf{y} = \begin{bmatrix} 4\\ 5\\ 6 \end{bmatrix} \]

we have:

\[ \mathbf{z} = \mathbf{x} + \mathbf{y}= \begin{bmatrix} 5\\ 7\\ 9 \end{bmatrix} \]

In NumPy:

x = np.arange(1, 4)
y = np.arange(4, 7)
z = x + y
z

array([5, 7, 9])

Compare the NumPy code with the Python code that involved lists and notice the differences. At least two are worth pointing:

• There is no explicit loop in the case of NumPy.
• The code is succinct and more readable.

We will soon see that there are also improvements to efficiency when using NumPy, especially when working with large arrays. Next we turn to some useful operations on vectors that can be performed with the help of NumPy.

Element-wise multiplication

Element-wise multiplication of two vectors is called the Hadamard product. The operator corresponding to it is \(\odot\). For example, given two vectors:

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \mathbf{y} = \begin{bmatrix} 4\\ 5\\ 6 \end{bmatrix} \]

we have:

\[ \mathbf{z} = \mathbf{x} \odot \mathbf{y}= \begin{bmatrix} 4\\ 10\\ 18 \end{bmatrix} \]

In NumPy:

x = np.arange(1, 4)
y = np.arange(4, 7)
z = x * y
z

array([ 4, 10, 18])

Scaling vectors

If \(\mathbf{x}\) is a vector, scaling it by a constant \(k\) is equivalent to element-wise multiplication by \(k\). For example, given

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} \]

we have:

\[ \mathbf{y} = 3 \mathbf{x} = \begin{bmatrix} 3\\ 6\\ 9 \end{bmatrix} \]

In NumPy:

x = np.arange(1, 4)
y = 3 * x
y

array([3, 6, 9])

Python Lists vs NumPy Arrays

A small detour. Let us now compare the difference in speed between Python lists and NumPy arrays. Consider a sequence of the first million positive integers. We wish to multiply each element by 2. Let us do it in two ways, one using Python lists and other using NumPy arrays, and observe the difference in speeds.

py_list = list(range(1, 1_000_000))
np_array = np.arange(1, 1_000_000)

%timeit [2 * x for x in py_list]

67.3 ms ± 11.7 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit 2 * np_array

796 µs ± 137 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

round((67.3 * 10 ** (-3)) / (796 * 10 ** (-6)))

85

We will look at some common operations on vectors and see how they can be achieved with NumPy.

Element-wise functions of vectors

Scaling a vector \(\mathbf{x}\) by a constant \(k\) can be seen as the outcome of the function \(f(x) = kx\) applied element-wise:

\[ \begin{bmatrix} f(x_1)\\ \vdots\\ f(x_d) \end{bmatrix} = \begin{bmatrix} kx_1\\ \vdots\\ k x_d \end{bmatrix} \]

NumPy extends this feature for any arbitrary function.

Example-1

For example, consider the function \(f(x) = x^2\). This can be applied element-wise:

\[ \begin{bmatrix} x_1^2\\ \vdots\\ x_d^2 \end{bmatrix} \]

Let us do this for:

\[ \begin{bmatrix} 1\\ 2\\ 3\\ 4 \end{bmatrix} \]

In NumPy:

x = np.arange(1, 5)
x ** 2

array([ 1,  4,  9, 16])

Example-2.1

As another example, consider \(f(x) = \log(x)\). This can also be applied element-wise:

\[ \begin{bmatrix} \log(x_1)\\ \vdots\\ \log(x_d) \end{bmatrix} \]

Let us do this for \(\begin{bmatrix}1 & 10 & 100 & 1000 & 10000 & 100000\end{bmatrix}^T\). Use base \(10\).

In NumPy:

x = np.array([1, 10, 100, 1000, 10_000, 100_000])
np.log10(x)

array([0., 1., 2., 3., 4., 5.])

Example 2.2

We can also use log to the base \(e\), the natural logarithm. For this, let us take a specific vector:

\[ \mathbf{x} = \begin{bmatrix} 1\\ e\\ e^2\\ e^3 \end{bmatrix} \]

and apply the function \(f(x) = \ln(x)\) element-wise.

In NumPy:

x = np.array([1, np.e, np.e ** 2, np.e ** 3])
np.log(x)

array([0., 1., 2., 3.])

Example 3

Just as we can scale a vector, we can also add a constant to each component. This is equivalent to applying the element-wise function \(f(x) = x + c\). Let us take the case of \(\begin{bmatrix}1 & 2 & 3\end{bmatrix}^T\) and \(c = 5\).

In NumPy:

x = np.arange(1, 4)
c = 5
x + c

array([6, 7, 8])

Example 4

Now for a slightly more involved example with \(f(x) = 5x - 2\) applied element-wise on the following vector:

\[ \mathbf{x} = \begin{bmatrix} -1\\ 0\\ 1\\ 2 \end{bmatrix} \]

In NumPy:

x = np.arange(-1, 3)
5 * x - 2

array([-7, -2,  3,  8])

Dot Product

The dot product between two vectors \(\mathbf{x}\) and \(\mathbf{y}\) is given as follows:

\[ z = \mathbf{x}^T \mathbf{y} = \mathbf{x} \cdot \mathbf{y} = \sum \limits_{j = 1}^{m} x_j y_j \]

Let us use the following vectors:

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3\\ 4\\ \end{bmatrix}, \mathbf{y} = \begin{bmatrix} -4\\ -5\\ -6\\ -7\\ \end{bmatrix} \]

In NumPy:

x = np.arange(1, 5)
y = np.arange(-4, -8, -1)
np.dot(x, y)

np.int64(-60)

Alternatively, we can do the following.

x @ y

np.int64(-60)

Since this resembles the mathematical expression quite accurately, we shall stick to this.

Vector of zeros or ones

On many occassions, we might want to create a NumPy array all of whose elements are zeros or ones or some other constant. Let us create the following vectors:

\[ \mathbf{0} = \begin{bmatrix} 0\\ 0\\ 0\\ 0\\ 0 \end{bmatrix}, \mathbf{1} = \begin{bmatrix} 1\\ 1\\ 1\\ 1\\ 1 \end{bmatrix} \]

In NumPy:

zeros = np.zeros(5)
zeros

array([0., 0., 0., 0., 0.])

ones = np.ones(5)
ones

array([1., 1., 1., 1., 1.])

What if we wanted to create a vector all of whose elements are equal to \(5\)?

fives = np.ones(5) * 5
fives

array([5., 5., 5., 5., 5.])

Norm of a vector

For a vector \(\mathbf{x}\):

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} \]

The \(L_2\) norm is defind as:

\[ ||\mathbf{x}||_2 = \sqrt{1^2 + 2^2 + 3^2} = \sqrt{14} \]

The \(L_1\) norm is defind as:

\[ ||\mathbf{x}||_1 = |1| + |2| + |3| = 6 \]

In NumPy:

# L2-norm
x = np.arange(1, 4)
norm = np.linalg.norm(x)
norm

np.float64(3.7416573867739413)

By default, the norm computed is the L2 norm. If we want to compute the L1 norm, we have to pass an additional argument to the function.

# L1-norm
x = np.arange(1, 4)
norm = np.linalg.norm(x, ord=1)
norm

np.float64(6.0)

Shape and dimension of a vector

Vectors are “one dimensional” arrays. So all vectors in NumPy have array-dimension equal to one. The term “array-dimension” here is defined for a NumPy array. It is different from the dimension used in the vector-space sense. The vector-space dimension can be obtaind by looking at the shape of the NumPy array. Let us explore these two ideas for:

\[ \mathbf{x} = \begin{bmatrix} 1\\ 2\\ 3\\ 4 \end{bmatrix} \]

# Shape
x = np.arange(1, 5)
x.shape

(4,)

# N-dim
x = np.arange(1, 5)
x.ndim

1