Linear Algebra
For the following notes, keep in mind the following 'tricks'.
What are we solving for?
Often in mathematics, a lot of assumptions are left unstated. It is not uncommon to not understand what is being asked, and so, it is beneficial to set up a generic framework to slot all the information given, so we can concentrate on solving the problem instead of understanding the language.
To do this, instead of passively reading a problem, we can ask questions to it.
The first question to ask is what are we solving for.
We may be given a problem in words, or in equations, or maybe just data.
We have to understand which are the independent variables, and which are the dependent or constrained variables.
An equation constrains the dependent variables based on the given value of the independent variables.
Prediction
A generic example is:
Here, one can think of
- \(f\) as the model, with parameters params
- \(x\) as the input, therefore the independent variable
- \(y\) as the output, which is constrained by the params and the input
To gain an understanding of the equation, it is advised to always draw a table.
Let us take a concrete example to get started:
We have \(y = f(x; \mbox{params})\):
| x | params | y | ... |
|---|---|---|---|
| 0 | p1 | 5 | ... |
| 1 | p1 | 6 | ... |
| 2 | p1 | 7 | ... |
For the above example, I do not give you any exposition.
Instead, try to imagine:
- What are we solving for?
- how many equations can we write?
- What is the solution?
Learning
Now, raising the difficulty, I modify the problem as follows:
We have
- \(\mbox{predicted}_i = f(x_i; \mbox{params})\)
- the training set: \(\{x_i, \mbox{expected}_i\}\) for 3 samples:
| x | expected | params | predicted | error |
|---|---|---|---|---|
| 0 | 4 | p1 | 5 | ? |
| 1 | 5 | p1 | 6 | ? |
| 2 | 6 | p1 | 7 | ? |
For the above example
- What are we solving for?
- how many equations can we write?
- What is a solution?
- How may solutions exist?
- What extra information, if any, do you need to answer the above questions?
If you were able to figure out the above, then you most likely understand the general shape of a problem, so let us move to the next 'trick'.
By Inspection
If there exists any shortcut to solving problems, it is to develop the superpower of by inspection.
There are a few 'tricks' to flex and train this power, we list the well-known ones below.
Verifying an answer is easier than solving to find the answer
It is sometimes very easy to dismiss solutions
For example, \(x \times (y + \mbox{log}(z)) = 53.26\)
Is \((x=0, y=8.29, z=\pi)\) a solution?
The most useful application of this is to find if a solution exists.
If two problems can be shown to be the same, solving one also solves the other
It is sometimes easier to just show that problem A is 'equal' to a solved problem B, and thus claim that A is also solved.