A matrix is a rectangular array of elements. These elements could be anything, but they are usually numbers when we discuss them as engineers. The core idea of matrices is ordering, by which we mean that the location of each element in our matrix tells us something about it. Compare a shopping list, which is an unordered vector where the location of each item on it doesn’t really matter; to a digital image, which is a matrix of numbers representing colours, where clearly the location matters a lot!
The above matrix is a 4$\times$3 matrix, ie it has four rows and three columns, so 12 elements in total.
We use matrices in mathematics and engineering because often we need to deal with several variables at once - eg coordinate vectors, as we saw in the previous chapter are a type of matrix which only has either a single column or a single row. In the following chapter on ‘linear transformations’ we’ll see a particular interpretation of the structure of a matrix relating to operations that transform vectors, but in this chapter we’re simply going to lay out the language and start building up a toolbox.
It turns out that many operations that are needed to be performed on coordinates of points are linear operations and so can be organised in terms of rectangular arrays of numbers (matrices). Then we find that matrices themselves can, under certain conditions, be added, subtracted and multiplied so that there arises a whole new set of algebraic rules for their manipulation
In general, a matrix ‘$\textit{A}$’ with dimensions of ($n \times m$) looks like:
It is convention to denote entries within a matrix $a_{ij}$, where $\textit{i}$ denotes the row and $\textit{j}$ denotes the column, also a capital letter is typically used to define the matrix itself.
The following sections describe methods for calculating various matrix operations; however, since the advent of modern computing, no one does this by hand any more … So you might wonder why we’re going over it! The answer is partly so that you know what’s going on in the machine and partly to make you appreciate how incredible computing power is!
3.1 Matrix Operations
3.1.1 Addition
It is possible to add two matrices together, but only if they have the same dimensions. We simply add the corresponding entries to form a new matrix of the same size:
If two matrices do not have the same dimensions they cannot be added, or we say the sum is ‘not defined’.
3.1.2 Multiplication or “Rows times cols”
When multiplying matrices, keep the following in mind: if the number of columns of the first matrix equals the number of rows of the second, then you can proceed. The process for multiplying is as follows: to find entry ($n$, $m$) of the resulting matrix, take row $n$ of the first matrix and column $m$ of the second matrix and then find the sum of the product of each pair of entries (ie the dot product). For example:
Where $\sum_{i=1}^{m}a_{1i}b_{i1}$ stands for $a_{11}b_{11} + a_{12}b_{21} + a_{13}b_{31} + \dots + a_{1n}b_{n1}$, etc.
Note that we must have $m = p$ such that the number of columns in the first matrix must equal the number of rows in the second; otherwise, we say the product is undefined.
The quick way to check whether a sequence of operations is allowed for matrices of different sizes is the following. Simply write down their dimensions as ``rows$\times$cols’’ and check that, wherever two adjacent matrices are multiplied, the adjacent dimensions are the same. Consider:
The operation for $A$ would work and the dimensions of the resulting matrix are those at the outer most of the operation, ie $(2\times7)$, whereas the operation for $B$ is undefined.
3.1.3 Scalar Multiplication
Another form of matrix multiplication is called scalar multiplication. This involves simply multiplying each entry of the matrix:
Another operation on matrices is the transpose. This reverses the rows and columns, or equivalently, reflects the matrix along the leading diagonal. The transpose of $A$ is normally written $A^{t}$, thus:
A square matrix is a matrix with the same number of rows as columns, ie $(n \times n)$. There are a number of special square matrices, however, a particularly important one is the ‘Identity Matrix’. This matrix fulfils a similar role to the number ‘1’ in calculations such that multiplying a matrix by the identity matrix results in no change to the matrix:
Whereby the size ($n\times n$) of the identity matrix is generally inherited from the other matrices involved in the operation unless otherwise stated.
The determinant is a property of a square matrix and is a scalar number defined by the entries within the matrix. It is very useful for calculating ‘how much larger’ (or smaller) a linear transformation has changed the original value (we’ll be talking about this in the next chapter, don’t worry for now!). The determinant of a matrix $A$ is denoted as either $\det(A)$ or $|A|$.
Calculating the determinant of a (2$\times$2) matrix
Calculating the determinant of a $(2\times 2)$ matrix is a trivial process. For a matrix $A$:
\[A =
\begin{pmatrix}
a & b \\
c & d
\end{pmatrix}\]
The determinant is simply the multiplication of $a \times d$ followed by the subtraction of $b \times c$:
Then, the minor $m_{ij}$, for each $i$ and $j$ is the determinant of the $(n-1 \times n-1)$-matrix obtained by deleting the $i^{th}$ row and the $j^{th}$ column. If you are left with a $(2\times 2)$-matrix the determinant is then taken. For example, in this notation:
The best way to remember this is as an ‘alternating’ pattern of positive and negative signs. Combining the minors from the previous example with this grid we get the cofactors:
Calculating the determinant of a (3$\times$3) matrix
In order to calculate the determinant of a $(3\times 3)$-matrix, choose any row or column. Then, multiply each entry by its corresponding cofactor, and add the three products. This gives the determinant.
Example - to show that any row or column may be used take matrix $A$:
Although it doesn’t matter which is chosen, it is common for the top row to be chosen. Note that it is not necessary to work out all the minors (or cofactors), just the three necessary!
The entire process above is neatly summarised in the adjacent diagram for a $3\times3$ matrix. It’s massively faster, so please make sure you understand what it’s asking; however, it’s still important that you understand the above so that you can tackle larger matrices.
Finding the determinant of an $(n\times n)$ matrix:
The procedure for large matrices is exactly the same as for a $(3\times 3)$ matrix: choose a row or column, multiply the entry by the corresponding cofactor and add them up. But of course each minor is itself the determinant of a $(n-1\times n-1)$-matrix so for example, in a $(4\times 4)$ determinant, it is necessary to do four $(3\times 3)$ determinants - quite a lot of work… but lucky you will never have to do this, thanks to computers!
3.5 Inverses
Let $A$ be an $(n\times n)$-matrix, and let $I$ be the $(n\times n)$ identity matrix. Sometimes, there exists a matrix $A^{-1}$ (called the inverse of $A$) with the property:
\[\begin{align}
AA^{-1}=I=A^{-1}A
\end{align}\]
3.5.1 Inverse of a (2x2) Matrix
From the above definition, we can simply write down the general case for a $2\times2$ matrix leaving the inverse matrix as four unknowns.
\[AA^{-1}=I
\qquad\text{then}\qquad
\begin{pmatrix}
a & b \\
c & d
\end{pmatrix}
\begin{pmatrix}
x_{11} & x_{12} \\
x_{21} & x_{22}
\end{pmatrix}=
\begin{pmatrix}
1 & 0 \\
0 & 1
\end{pmatrix}\]
And now we’re simply looking to find the 4 values of $x$ in terms of $a, b, c, \&\ d$.
First we can write down the four simultaneous equations generated by applying the matrix multiplication.
To check that our results is right, we can just multiply it back through by the original matrix:
\[\begin{align}
A^{-1} \times A &= \frac{1}{ad-bc}
\begin{pmatrix}
d & -b \\
-c & a
\end{pmatrix}
\begin{pmatrix}
a & b \\
c & d
\end{pmatrix}
\\\\
&= \frac{1}{ad-bc}
\begin{pmatrix}
da-bc & db-bd \\
-ca+ac & -cb+ad
\end{pmatrix}
\\\\
&= \frac{1}{ad-bc}
\begin{pmatrix}
ad-bc & 0 \\
0 & ad-bc
\end{pmatrix}
\\\\
&=
\begin{pmatrix}
1 & 0 \\
0 & 1
\end{pmatrix}
= I
\end{align}\]
3.5.2 Inverse of (3x3) matrices (or higher)
We can follow the same process of simultaneous equation solving for $3\times3$ matrices and generate an explicit equation for the inverse; however the equation is complex, so it’s typically easier to remember an algorithm for its calculation rather than finding it directly and this algorithm scales to an arbitrary $n\times n$ matrix.
Recall the definition of a minor: given an $(n\times n)$-matrix, $A$, the minor $m_{ij}$ is the determinant of the $(n-1\times n-1)$-matrix by omitting the $i^{th}$ row and $j^{th}$ column. Then the cofactor is then the minor multiplied by the `alternating’ positive and negative patterns.
The same procedure works for $(n\times n)$ matrices:
Work out minors
Put in the $-$ signs to form the cofactors
Take the transpose
Divide by the determinant
Furthermore, an $(n\times n)$ matrix has an inverse if and only if the determinant is not zero. So it’s a good idea to calculate the determinant first, just to see if the rest of the procedure is necessary.
3.6 Linear Systems
The method of solving simultaneous equations most school students are aware of involves re-arranging one equation such that it may be `inserted’ into the other. Try to solve the two equations:
By rearranging we can calculate the values of $x$ and $y$:
\[\begin{gather}
y = 3 - 2x \quad\Longrightarrow\quad 5x + 3(3 - 2x) = 7 \\
\therefore\qquad x = 2 \qquad \text{and} \qquad y = -1
\end{gather}\]
However, we can use matrices to provide a more systematic approach to solving simultaneous equations (despite this particular example being solved very simply without using matrices!). To do so we can re-write the equations in a slightly different way:
\[\begin{align}
\begin{pmatrix}
1 & 0 \\
0 & 1
\end{pmatrix}
\begin{pmatrix}
x \\
y
\end{pmatrix}
&=
\begin{pmatrix}
(3\times3) + (-1\times7) \\
(-5\times3) + (2\times7)
\end{pmatrix}
\\\\
\begin{pmatrix}
x \\
y
\end{pmatrix}
&=
\begin{pmatrix}
2 \\
-1
\end{pmatrix}
\end{align}\]
and so $x = 2$ and $y = -1$ as required. So, provided we can work out the inverse of the matrix of coefficients, we can solve simultaneous equations. Finding efficient algorithms for inverting matrices is at the heart of computer science!
3.6.1 Larger Systems
The same thing works with 3 equations and $x$, $y$ and $z$. Suppose we have:
A triangular matrix is a square matrix that is either lower triangular or upper triangular. In a lower triangular matrix, all the entries above the main diagonal are zero. In a upper triangular matrix, all the entries below the main diagonal are zero.
This is the longest chapter in the DE1-MEM course, but in many ways I hope you can agree that it’s the most straightforward as it’s just about a set of tools and conventions used for manipulating matrices. Handy phrases like ‘rows times cols’ will help you remember whether a multiplication operation is defined, but please rest assured that in the exam (and in your career!) you will never be asked to simply invert monstrous $5\times 5$ matrices by hand, as this would only test your calculator skills.
The entire process above is neatly summarised in the adjacent diagram for a $3\times3$ matrix. It’s massively faster, so please make sure you understand what it’s asking; however, it’s still important that you understand the above so that you can tackle larger matrices.