Introduction to Numpy

Python lists:

  • are very flexible
  • don't require uniform numerical types
  • are very easy to modify (inserting or appending objects).

However, flexibility often comes at the cost of performance, and lists are not the ideal object for numerical calculations.

This is where Numpy comes in. Numpy is a Python module that defines a powerful n-dimensional array object that uses C and Fortran code behind the scenes to provide high performance.

The downside of Numpy arrays is that they have a more rigid structure, and require a single numerical type (e.g. floating point values), but for a lot of scientific work, this is exactly what is needed.

The Numpy module is imported with:

In [ ]:
#  import numpy

Although in the rest of this course, and in many packages, the following convention is used:

In [ ]:
import numpy as np

This is because Numpy is so often used that it is shorter to type np than numpy.

Creating Numpy arrays

The easiest way to create an array is from a Python list, using the array function:

In [ ]:
a = np.array([10, 20, 30, 40])
In [ ]:
print(a)

Numpy arrays have several attributes that give useful information about the array:

In [ ]:
a.ndim  # number of dimensions
In [ ]:
a.shape  # shape of the array
In [ ]:
a.dtype  # numerical type
In [ ]:
a.size # number of elements in array
In [ ]:
b = a.copy() # Create a copy of the array
print(b)
In [ ]:
a.tolist() # Convert a numpy array to a Python list

Note: Numpy arrays actually support more than just one integer type and one floating point type - they support signed and unsigned 8-, 16-, 32-, and 64-bit integers, and 16-, 32-, and 64-bit floating point values.

There are several other ways to create arrays. For example, there is an arange function that can be used similarly to the built-in Python range function, with the exception that it can take floating-point input:

In [ ]:
np.arange(10) # Create an array from 0 to 9
In [ ]:
np.arange(3, 12, 2) # Create an array from 3 to 11 in steps of 2
In [ ]:
np.arange(1.2, 4.4, 0.1) # Create an array from 1.2 to 4.3 in steps of 0.1

Another useful function is linspace, which can be used to create linearly spaced values between and including limits:

In [ ]:
np.linspace(11., 12., 11)

and a similar function can be used to create logarithmically spaced values between and including limits:

In [ ]:
np.logspace(1., 4., 7)

Finally, the zeros and ones functions can be used to create arrays intially set to 0 and 1 respectively:

In [ ]:
np.zeros(10)
In [ ]:
np.ones(5)

Combining arrays

Numpy arrays can be combined numerically using the standard +-*/** operators:

In [ ]:
x = np.array([1,2,3])
y = np.array([4,5,6])
In [ ]:
x + 2 * y
In [ ]:
x ** y

Note that this differs from lists:

In [ ]:
x = [1,2,3]
y = [4,5,6]
In [ ]:
x + 2 * y

You can also apply a function to an entire array:

In [ ]:
x = np.array([7,8,9])
z = np.sin(x)
print(z)

Exercise 1

Create an array which contains 11 values logarithmically spaced between $10^{-20}$ and $10^{-10}$.

In [ ]:
# your solution here

Create an array which contains the value 2 repeated 10 times (hint: there are two quick ways to do that – one is by multiplying a list, one by multiplying an array)

In [ ]:
# your solution here

Try using np.empty(10) and compare the results to np.zeros(10). Also compare to what the people next to you got for np.empty. What do you think is going on?

In [ ]:
# your solution here

Create an array containing 5 times the value 0, as a 32-bit floating point array (hint: have a look at the docs for np.zeros; the type you'd be looking for is called float32)

In [ ]:
# your solution here

Accessing and Slicing Arrays

Similarly to lists, items in arrays can be accessed individually:

In [ ]:
x = np.array([9,8,7])
In [ ]:
x[0]
In [ ]:
x[1]

and arrays can also be sliced by specifiying the start and end of the slice (where the last element is exclusive):

In [ ]:
y = np.arange(10)
In [ ]:
y[0:5]

optionally specifying a step:

In [ ]:
y[0:10:2]

As for lists, the start, end, and step are all optional, and default to 0, len(array), and 1 respectively:

In [ ]:
y[:5]
In [ ]:
y[::2]

Exercise 2

Given an array x with 10 elements, find the array dx containing 9 values where dx[i] = x[i+1] - x[i]. Do this without loops!

In [ ]:
# your solution here

Multi-dimensional arrays

Numpy can be used for multi-dimensional arrays:

In [ ]:
x = np.array([[1.,2.],[3.,4.]])
x
In [ ]:
x.ndim # Number of dimensions
In [ ]:
x.shape # (rows, columns)

You can refer to (or set) specific elemens as:

In [ ]:
x[0,1] # The first number is the row number
       # The second number is the column number 
       # (remember python counts from 0)
In [ ]:
x[0,1] = -1.0
x

If you just want to do something with the numbers in the second row, then you can just refer to it using a single index:

In [ ]:
x[1]
In [ ]:
y = np.ones([3,2,3])  # ones takes the shape of the array, not the values
In [ ]:
y # Three sets of 2 lines by 3 columns
In [ ]:
y.shape

If you want to fill the array with a specific value:

In [ ]:
y.fill(100.0)
y

Beware of type coersion!:

In [ ]:
k = np.array([0,1,2,3,4,5]) # This is defined as an integer (int32) array
k
In [ ]:
k[0] = 10.546 # Assigning a float into it will truncate the decimal part!
k

Using np.linalg.inv() we can calculate the inverse of a square array, interpreted as a matrix:

In [ ]:
a=np.array([[1,2],[3,4]])
In [ ]:
np.linalg.inv(a)

Example: The solution of the system of equations

x   + 2*y = 5
3*x + 4*y = 2

can be found using np.dot(), which yields the matrix product for 2-D arrays

In [ ]:
np.dot(np.linalg.inv(a),[5,2])

Multi-dimensional arrays can be sliced differently along different dimensions:

In [ ]:
z = np.ones([6,6,6])
In [ ]:
zz=z[::3, 1:4, :]

print(zz)
zz.shape

Functions

In addition to an array class, Numpy contains a number of vectorized functions, which means functions that can act on all the elements of an array, typically much faster than could be achieved by looping over the array.

For example:

In [ ]:
theta = np.linspace(0., 2. * np.pi, 10)
In [ ]:
theta
In [ ]:
np.sin(theta)

Another useful package is the np.random sub-package, which can be used to genenerate random numbers fast:

In [ ]:
# uniform distribution between 0 and 1
np.random.random(10)
In [ ]:
# 10 values from a gaussian distribution with mean 3 and sigma 1
np.random.normal(3., 1., 10)

Another very useful function in Numpy is numpy.loadtxt which makes it easy to read in data from column-based data. For example, given the following file:

In [ ]:
%cat data/autofahrt.txt

We can either read it in using a single multi-dimensional array:

In [ ]:
data = np.loadtxt('data/autofahrt.txt',skiprows=2)
data

Or we can read the individual columns:

In [ ]:
time,ax,ay,az = np.loadtxt('data/autofahrt.txt', skiprows=2, unpack=True)
In [ ]:
time[:10]
In [ ]:
ax[:10]

There are additional options to skip header rows, ignore comments, define delimiters, or read only certain columns. See the documentation for more details.

Masking

The index notation [...] is not limited to single element indexing, or multiple element slicing, but one can also pass a discrete list/array of indices:

In [ ]:
x = np.array([1,6,4,7,9,3,1,5,6,7,3,4,4,3])
x[[1,2,4,3,3,2]]

which is returning a new array composed of elements 1, 2, 4, etc from the original array.

Alternatively, one can also pass a boolean array of True/False values, called a mask, indicating which items to keep:

In [ ]:
x[np.array([True, False, False, True, True, True, False, False, True, True, True, False, False, True])]

Now this doesn't look very useful because it is very verbose, but now consider that carrying out a comparison with the array will return such a boolean array:

In [ ]:
x > 3.4

It is therefore possible to extract subsets from an array using the following simple notation:

In [ ]:
x[x > 3.4]

Conditions can be combined:

In [ ]:
x[(x > 3.4) & (x < 5.5)]

Of course, the boolean mask can be derived from a different array to x as long as it is the right size:

In [ ]:
x = np.linspace(-1., 1., 14)
y = np.array([1,6,4,7,9,3,1,5,6,7,3,4,4,3])
In [ ]:
y[(x > -0.5) & (x < 0.4)]

Since the mask itself is an array, it can be stored in a variable and used as a mask for different arrays:

In [ ]:
keep = (x > -0.5) & (x < 0.4)
x_new = x[keep]
y_new = y[keep]
In [ ]:
x_new
In [ ]:
y_new

A mask can also appear on the left hand side of an assignment:

In [ ]:
y[y > 5] = 0.
In [ ]:
y

Exercise 3

The data/munich_temperatures_average_with_bad_data.txt data file gives the temperature in Munich every day for several years:

In [ ]:
!head data/munich_temperatures_average_with_bad_data.txt  # shows the 10 first lines of a file

Read in the file using np.loadtxt. The data contains bad values, which you can identify by looking at the minimum and maximum values of the array. Use masking to get rid of the bad temperature values.

In [ ]:
# your solution here
year, temp = np.loadtxt('data/munich_temperatures_average_with_bad_data.txt', unpack=True)
In [ ]:
year
In [ ]:
mask = (temp > -50.0) & (temp < 50.0)
temp_clean = temp[mask]
year_clean = year[mask]

min(temp_clean), max(temp_clean)
len(year)-len(year_clean)
In [ ]: