Matrices+ 01 - Matrix as Operator: UNIZOR.COM - Math+ & Problems - Matrices

Notes to a video lecture on http://www.unizor.com

Matrices+ 01
Matrix as Operator

In the beginning matrices were just tables of real numbers with m rows and n columns, we called them m⨯n matrices.
We knew how to add two matrices with the same numbers of rows and columns, how to multiply a matrix by a scalar number and how to multiply an m⨯n matrix by an n⨯k matrix.

Here is a short recap.

Addition of matrices
Let A_m⨯n be an m⨯n matrix with elements a_i,j, where the first index signifies the row, where this element is located and the second index is a column.
Let B_m⨯n be another m⨯n matrix with elements b_i,j.
Then a new matrix C=A+B is an m⨯n matrix with elements c_i,j=a_i,j+b_i,j.

Multiplication of a matrix by a scalar
Let s be any real number (scalar) and A_m⨯n be an m⨯n matrix with elements a_i,j.
Multiplication of s by A_m⨯n produces a new matrix B_m⨯n with elements b_i,j=s·a_i,j.
Obviously, since the product of numbers is commutative and associative, the product of a matrix by a scalar is commutative, and the product of a matrix by two scalars is associative.

Product of two matrices
Let A_m⨯n be an m⨯n matrix with elements a_i,j.
Let B_n⨯k be an n⨯k matrix with elements b_i,j.
It's important for a definition of a product of matrices that the number of columns in the first matrix A_m⨯n equals to a number of rows in the second B_n⨯k (in our case this number is n.)
The product C_m⨯k of A_m⨯n and B_n⨯k is m⨯k matrix with elements c_p,q calculated as sums of products of n elements of p^th row of matrix A_m⨯n by n elements of q^th column of matrix B_n⨯k
∀p∈[1,m], ∀q∈[1,k]:
c_p,q = Σ_[1≤t≤n]a_p,t·b_t,q
It's important to note that, generally speaking, multiplication of two matrices IS NOT commutative, see Problem C below.
The product of three matrices, however, is associative, see Problem B below.

For our purposes we will only consider "square" matrices with the same number of rows and columns n and n-dimensional Euclidean vector space of sequences of n real numbers organized in a row (row-vector, which can be viewed as 1⨯n matrix) or in a column (column-vector, which can be viewed as n⨯1 matrix).

What happens if we multiply an n⨯n matrix by a column-vector, which we can consider as an n⨯1 matrix, according to the rules of multiplication of two matrices?
In theory, the multiplication is possible because the number of columns in the first matrix n equals to the number of rows in the second one (the same n) and, according to the rules of multiplication, the resulting matrix should have the number of rows of the first matrix n and the number of columns of the second one, that is 1. So, the result is an n⨯1 matrix, that is a column-vector.

As we see, multiplication of our n⨯n matrix by a column-vector with n rows results in another column-vector with n row.
In other word, multiplication by n⨯n matrix on the left transforms one column-vector into another, that is this multiplication represents an operation in n-dimensional vector space of column-vectors. The n⨯n matrix itself, therefore, acts as an operator in the vector space of column-vectors of n components.

Similar operations can be considered with row-vectors and their multiplication by matrices. The only difference is the order of this multiplication.
In a case of column-vectors we multiplied n⨯n matrix by n⨯1 column-vector getting another n⨯1 column-vector.
In case of row-vectors, we change the order and multiply a 1⨯n row-vector by n⨯n matrix on the right, getting another 1⨯n row-vector.
Therefore, an n⨯n matrix can be considered an operator in a vector space of row-vectors, if we apply the multiplication by matrix from the right.

Let's examine the properties of the transformation of n⨯1 column-vectors by multiplying them by n⨯n matrices.

Problem A
Prove that multiplication by an n⨯n matrix is a linear transformation in the n-dimensional Euclidean space ℝⁿ of n⨯1 column-vectors.
(a) ∀u∈ℝⁿ, ∀ K∈ℝ, ∀A_n⨯n:
A·(K·u) = K·(A·u) = (K·A)·u
(b) ∀u,v∈ℝⁿ, ∀A_n⨯n:
A·(u+v) = A·u + A·v
(c) ∀u∈ℝⁿ, ∀A_n⨯n,B_n⨯n:
(A+B)·u = A·u + B·u

Hint A
The proof is easily obtained straight from the definition of matrix multiplications.


Problem B
Prove that consecutive multiplication by two n⨯n matrices is associative in the n-dimensional Euclidean space ℝⁿ of n⨯1 column-vectors.
∀u∈ℝⁿ, ∀A_n⨯n,B_n⨯n:
A·(B·u) = (A·B)·u

Hint B
It's all about changing of an order of summation.
Let's demonstrate it for n=2.

Given two matrices A and B and a column-vector u.

Matrix A

a1,1	a1,2
a2,1	a2,2

Matrix B

b1,1	b1,2
b2,1	b2,2

Column-vector u

u1

u2

Let v=B·u and w=A·(B·u)=A·v
Then column-vector v's components are
v₁ = b_1,1·u₁ + b_1,2·u₂
v₂ = b_2,1·u₁ + b_2,2·u₂

The components of vector w are
w₁ = a_1,1·v₁ + a_1,2·v₂ =
= a_1,1·(b_1,1·u₁ + b_1,2·u₂) +
+ a_1,2·(b_2,1·u₁ + b_2,2·u₂) =
= a_1,1·b_1,1·u₁ + a_1,1·b_1,2·u₂ +
+ a_1,2·b_2,1·u₁ + a_1,2·b_2,2·u₂ =
= (a_1,1·b_1,1 + a_1,2·b_2,1)·u₁ +
+ (a_1,1·b_1,2 + a_1,2·b_2,2)·u₂

w₂ = a_2,1·v₁ + a_2,2·v₂ =
= a_2,1·(b_1,1·u₁ + b_1,2·u₂) +
+ a_2,2·(b_2,1·u₁ + b_2,2·u₂) =
= a_2,1·b_1,1·u₁ + a_2,1·b_1,2·u₂ +
+ a_2,2·b_2,1·u₁ + a_2,2·b_2,2·u₂ =
= (a_2,1·b_1,1 + a_2,2·b_2,1)·u₁ +
+ (a_2,1·b_1,2 + a_2,2·b_2,2)·u₂

Let's calculate the product of two matrices A·B
Matrix A·B:

a1,1·b1,1+a1,2·b2,1	a1,1·b1,2+a1,2·b2,2
a2,1·b1,1+a2,2·b2,1	a2,1·b1,2+a2,2·b2,2

Notice that coefficients at u₁ and u₂ in expression for w₁ above are the same as elements of the table (A·B) of the first row.
Analogously, coefficients at u₁ and u₂ in expression for w₂ above are the same as elements of the table (A·B) of the second row.
That means that, if we multiply matrix (A·B) by a column-vector u, we will get the same vector w as above.
That proves the associativity for a two-dimensional case.

Problem C
Prove that consecutive multiplication by two n⨯n matrices, generally speaking, IS NOT commutative.
That is, in general,
(A·B)·u) ≠ (B·A)·u

Proof C
To prove it, it is sufficient to present a particular case when a product of two matrices is not commutative.

Consider matrix A

1	2
3	4

Matrix B

−1	2
−3	4

Then matrix A·B will be

−7	10
−15	22

Reverse multiplication B·A will be

5	6
9	10

As you see, matrices A·B and B·A are completely different, and, obviously, their product with most vectors will produce different results.
Take a vector with components (1,0), for example. Matrix A·B will transform it into (−7,−15), while matrix B·A will transform it into (5,9).
End of Proof.

Vectors+ 11 - 2D Vector · Complex scalar: UNIZOR.COM - Math+ & Problems ...

Notes to a video lecture on http://www.unizor.com

Vectors+ 11
2D Vector · Complex scalar

We know that a two-dimensional vector multiplied by a real scalar changes its length proportionally, but the direction remains the same for a positive multiplier or changes to opposite for a negative multiplier.
Let's examine how vectors on a two-dimensional plane are affected if multiplied by a non-zero complex scalar.

Our first task is to define a multiplication operation of a vector by a complex number.
To accomplish this, let's recall that any complex number a+i·b, where both a and b are real numbers and i²=−1, can be represented by a vector in a two-dimensional Euclidean space (that is, on the coordinate plane) with abscissa a and ordinate b.
This representation establishes one-to-one correspondence between vectors on a plane and complex numbers.

Using this representation, let's define an operation of multiplication of a vector on a coordinate plane {a,b} by a complex number z=x+i·y as follows:
1. Find a complex number that corresponds to our vector. So, if a vector has coordinates {a,b}, consider a complex number a+i·b.
2. Multiply this complex number by a multiplier z=x+i·y using the rules of multiplication of complex numbers. This means
(a+i·b)·z = (a+i·b)·(x+i·y) =
= a·x+a·i·y+i·b·x+i·b·i·y =
= (a·x−b·y) + i·(a·y+b·x)
3. Find the vector that corresponds to a result of multiplication of two complex number in a previous step. This vector should have abscissa a·x−b·y and ordinate a·y+b·x.
4. Declare the vector in step 3 as a result of an operation of multiplication of the original vector {a,b} by a complex multiplier z=x+i·y. So, the result of multiplication of vector {a,b} by a complex multiplier z=x+i·y is a vector {a·x−b·y,a·y+b·x}

Now we have to examine the geometric aspect of this operation.
For this, let's represent a multiplier z=x+i·y as
z = √x²+y²·(x/√x²+y² + i·y/√x²+y²)

Two numbers, x/√x²+y² and y/√x²+y², are both in the range from −1 to 1 and a sum of their squares equals to 1.
Find an angle φ such that
cos(φ)=x/√x²+y² and
sin(φ)=y/√x²+y².
Now the multiplier looks like
z = |z|·[cos(φ) + i·sin(φ)]
where |z| = √x²+y²

Using this representation, the product of a vector {a,b} by multiplier z=x+i·y looks like
{a·x−b·y,a·y+b·x} = {a',b'}
where
a' = |z|·[a·cos(φ)−b·sin(φ)] and
b' = |z|·[a·sin(φ)+b·cos(φ)]

The geometric meaning of the transformation from vector {a,b} to vector {a',b'} is a rotation of the vector by angle φ.
Here is why.
The length of a vector {a,b} is L=√a²+b².
If the angle this vector makes with an X-axis is α, the abscissa and ordinate of our vector can be expressed as
a = L·cos(α)
b = L·sin(α)

Using this representation, let's express the coordinates of a vector {a',b'} obtained as a result of multiplication of the original vector {a,b} by a complex
z=x+i·y=|z|·[cos(φ) + i·sin(φ)]
in terms of L and α.

a'=|z|·[L·cos(α)·cos(φ)−L·sin(α)·sin(φ)]
and
b'=|z|·[L·cos(α)·sin(φ)+L·sin(α)·cos(φ)]

Recall from Trigonometry:
cos(α+φ)=cos(α)·cos(φ)−sin(α)·sin(φ)
sin(α+φ)=cos(α)·sin(φ)+sin(α)·cos(φ)

Now you see that
a'=|z|·L·cos(α+φ)
b'=|z|·L·sin(α+φ)

So, the multiplication of a vector {a=L·cos(α),b=L·sin(α)} by a complex number z=x+i·y=|z|·[cos(φ)+i·sin(φ)], can be interpreted as changing the length of an original vector by a factor |z| and a rotation by angle φ.

Vectors+ 10 Complex Hilbert Space: UNIZOR.COM - Math+ & Problems - Vectors

Notes to a video lecture on http://www.unizor.com

Vectors+ 10
Complex Hilbert Spaces

As we know, a Hilbert space consists of
(a) an abstract vector space V with operation of addition
∀v₁,v₂∈V: v₁+v₂∈V
(b) a scalar space S of numbers (we consider only two cases: S=ℝ - a set of all real numbers or S=ℂ - a set of all complex numbers) with operation of multiplication of a scalar from this set S by a vector from vector space V
∀λ∈S and ∀v∈V: λ·v∈V
(c) an operation of a scalar product of two vectors from V denoted as ⟨v₁,v₂⟩, resulting in a scalar from S
∀v₁,v₂∈V: ⟨v₁,v₂⟩∈S

Operations of addition of two vectors, resulting in a vector, multiplication of a vector by a scalar, resulting in a vector, and scalar product of two vectors, resulting in a scalar, must satisfy certain set of axioms presented in lecture Vectors 08 of this part of a course, including commutative, associative and distributive laws.

In all the previously considered cases our scalar space S was a set of all real numbers ℝ.
Now we will consider a set of all complex numbers ℂ as a scalar space S and examine the validity of our axioms as applied to a simple vector space.
By all accounts this should not present any problems since arithmetic operations with complex numbers are similar to those with real numbers.
There is, however, one complication.

One of the axioms of a scalar product was:
For any vector a from a vector space V, which is not a null-vector, its scalar product with itself is a positive real number
∀a∈V, a≠0: ⟨a,a⟩ > 0
This axiom is needed to introduce a concept of a length (or a magnitude, or a norm) of any vector as
||a|| = √⟨a,a⟩

Consider now a simple case of a vector space V - a set of all complex numbers ℂ with a scalar space S being a set of complex numbers ℂ as well.
We will use the word vector as an element of a vector space V and the word scalar as an element of a scalar space S, but both spaces are sets of all complex numbers ℂ.

Addition of two vectors (complex numbers) and multiplication of a vector (a complex number) by a scalar (also a complex number) are defined, as regular operations with complex numbers.
Scalar product of two vectors (two complex numbers) is defined as their regular operation of multiplication.
Let's check how valid our definition of a length of a vector is in this case with all the axioms previously introduced for a set of all real numbers ℝ being the scalar space S.

Vector a from V (a complex number) can be expressed as a₁+i·a₂, where a₁ and a₂ are also vectors from V, but belong to a subset of only real numbers, while i is an imaginary unit √−1, an element of a scalar space S (also a set of all complex numbers).

Let's calculate a scalar product of vector a with itself using the old laws of operations previously defined
⟨a₁+i·a₂,a₁+i·a₂⟩ =
[since a₁ and a₂ are real numbers]
= a₁²+2i·a₁·a₂+i²·a₂² =
= a₁²+2i·a₁·a₂−a₂²
The problem is, this is not a positive number since it contains an imaginary part.
So, the set of axioms we introduced before for a scalar space being a set of all real numbers ℝ is contradictory in case of complex numbers ℂ as a scalar space for a simple case above.

To overcome this problem, we have to revise our axioms in such a way that they will hold for examples like above for complex scalars, while being held as well for real numbers as scalars, since real numbers are a subset of complex ones.

There are two small changes in the axioms for a scalar product that we introduce to solve this problem.
Let a and b be two vectors and λ - a complex number as a scalar.
(1) λ·⟨a,b⟩ = ⟨λ·a,b⟩ = ⟨a,λ·b⟩
(2) ⟨a,b⟩ = ⟨b,a⟩
where horizontal bar above a complex number means complex conjugate number, that is a number x−i·y for a number x+i·y and a number x+i·y for a number x−i·y.
Incidentally, a conjugate to imaginary unit i (that is, 0+i·1) is −i (that is, 0−i·1).

First of all, if a complex number x+i·y is, actually, a real one (that is, if y=0), its conjugate number is the same as the original. So, in case of a scalar space being a set of all real numbers ℝ these axioms are exactly the same as the old previously accepted ones.

Secondly, for complex scalars these modified axioms solve the problem in a simple case of V=ℂ and S=ℂ mentioned before for a vector a=a₁+i·a₂, where a₁ and a₂ are from a subset of only real numbers.

Let's calculate a scalar product using the modified rules.
⟨a₁+i·a₂,a₁+i·a₂⟩ =
[using distributive law]
= ⟨a₁,a₁⟩ + ⟨a₁,i·a₂⟩ +
+ ⟨i·a₂,a₁⟩ + ⟨i·a₂,i·a₂⟩ =
= a₁² + ⟨a₁,i·a₂⟩ +
+ ⟨i·a₂,a₁⟩ + ⟨a₂·−i·i·a₂⟩ =
[since −i²=1 and addition is commutative]
= a₁² + a₂² + ⟨a₁,i·a₂⟩ + ⟨i·a₂,a₁⟩ =
[using a newly modified rule]
λ·⟨a·b⟩ = ⟨λ·a,b⟩ = ⟨a,λ·b⟩]
= a₁² + a₂² + ⟨−i·a₁,a₂⟩ + ⟨i·a₂,a₁⟩ =
= a₁² + a₂² + (−i)·⟨a₁,a₂⟩ + i·⟨a₂,a₁⟩ =
[since a₁ and a₂ are vectors from a subset of real numbers, their scalar product is commutative, and the last two terms cancel each other]
= a₁² + a₂²
which is a positive real number for any non-zero complex number.

This expression fully corresponds to a concept of an absolute value of a complex number a₁+i·a₂ and to a concept of a length of a vector in two-dimensional Euclidean space with abscissa a₁ and ordinate a₂ used as graphical representation of a complex number a₁+i·a₂.

Moreover, similar calculations can be applied to N-vector of complex numbers as a vector space with a set of all complex numbers as a scalar space.
Indeed, consider for brevity a case of N=2 since the more general case of any N is totally analogous.
The element of our two-dimensional vector space is a pair {a,b}, where both components are complex numbers.
Its scalar product with itself is defined as
⟨{a,b},{a,b}⟩ = (a·a) + (b·b)
and each term on the right of this equation, as we determined, is a positive real number.

As we see, a small modification of the axioms of scalar product in case of complex scalars solves the problem and, at the same time, does not change anything we knew about scalar product with real scalars.

As we did before, we can say that in case of a complex scalar space a scalar product of a non-null vector by itself is positive and the square root of it is its length or magnitude or norm
||a||² = ⟨a,a⟩

Problem A
Prove the Cauchy-Schwarz-Bunyakovsky inequality for an abstract vector space V and a complex scalar space S=ℂ
∀a,b∈V: |⟨a·b⟩|² ≤ ⟨a·a⟩·⟨b·b⟩
where absolute value of any complex number Z=X+i·Y is defined as
|Z|² = X² + Y² = Z·Z

Proof A
Consider any non-zero scalar (complex number) x and non-negative (by axiom) scalar product of vector a+xb by itself
0 ≤ ⟨a+x·b,a+x·b⟩
Use distributive properties to open all parenthesis
0 ≤ ⟨a,a⟩+⟨a,x·b⟩+⟨x·b,a⟩+⟨x·b,x·b⟩
Using the modified commutative rules introduced in this lecture, we can transform individual members of this expression as follows
⟨a,x·b⟩ = x·⟨a,b⟩
⟨x·b,a⟩ = x·⟨b,a⟩
⟨x·b,x·b⟩ = x·x·⟨b,b⟩

Now our inequality looks like
0 ≤ x·x·⟨b,b⟩+x·⟨a,b⟩+x·⟨b,a⟩+⟨a,a⟩

This inequality is true for any complex number x.
If vector b equals a null-vector, the inequality is held because ⟨b,b⟩=0 and ⟨a,b⟩=0 (see Problem A of lecture Vectors 08 of this part of a course).
Assume, vector b is not a null-vector, and let's see how our inequality looks for specific value of x:
x = −⟨a,b⟩/⟨b,b⟩

Let's evaluate each member of the inequality above.
x·x·⟨b,b⟩ = ⟨a,b⟩·⟨a,b⟩/⟨b,b⟩
x·⟨a,b⟩ = −⟨a,b⟩·⟨a,b⟩/⟨b,b⟩
x·⟨a·b⟩ = −⟨a,b⟩·⟨a,b⟩/⟨b,b⟩

Putting these values into our inequality and cancelling plus and minus of the same numbers, we obtain
0 ≤ −⟨a,b⟩·⟨a,b⟩/⟨b,b⟩ + ⟨a,a⟩
Multiplying by positive ⟨b,b⟩ and separating members into different sides of an inequality, we obtain
⟨a,b⟩·⟨a,b⟩ ≤ ⟨a,a⟩·⟨b,b⟩
or, since |Z|²=Z·Z for any complex Z,
|⟨a,b⟩|² ≤ ⟨a,a⟩·⟨b,b⟩
End of proof.

Vectors+ 09 Example of Hilbert Space: UNIZOR.COM - Math+ & Problems - Ve...

Notes to a video lecture on http://www.unizor.com

Vectors+ 09
Examples of Hilbert Spaces

Let's illustrate our theory of Hilbert spaces with a few examples.

Example 1

Consider a set V of all polynomials of real argument x defined on a segment [0,1].
It's a linear vector space with any polynomial acting as a vector in this space because all the previously mentioned axioms for an abstract vector space are satisfied:
(A1) Addition of any two polynomials a(x) and b(x) is commutative
∀a(x),b(x)∈V: a(x) + b(x) = b(x) + a(x)
(A2) Addition of any three polynomials a(x), b(x) and c(x) is associative
∀a(x),b(x),c(x)∈V: [a(x)+b(x)]+c(x) =
= a(x)+[b(x)+c(x)]
(A3) There is one polynomial that is equal to 0 for any argument in segment [0,1] called null-polynomial, denoted 0(x) (that is, 0(x)=0 for any x of a domain [0,1]), with a property of not changing the value of any other polynomial a(x) if added to it
∀a(x)∈V: a(x) + 0(x) = a(x)
(A4) For any polynomial a(x) there is another polynomial called its opposite, denoted as −a(x), such that the sum of a polynomial and its opposite equals to null-polynomial (that is a polynomial equaled to zero for all arguments)
∀a(x)∈V ∃−a(x)∈V: a(x)+(−a(x))=0(x)

(B1) Multiplication of any scalar (element of a set of all real numbers) α by any polynomial a(x) is commutative
∀a(x)∈V, ∀real α: α·a(x) = a(x)·α
(B2) Multiplication of any two scalars α and β by any polynomial a(x) is associative
∀a(x)∈V, ∀real α,β:
(α·β)·a(x) = α·(β·a(x))
(B3) Multiplication of any polynomial by scalar 0 results in null-polynomial
∀a(x)∈V: 0·a(x) = 0(x)
(B4) Multiplication of any polynomial by scalar 1 does not change the value of this polynomial a(x)
∀a(x)∈V: 1·a(x) = a(x)
(B5) Multiplication is distributive relatively to addition of polynomials. ∀a(x),b(x)∈V, ∀real α:
α·(a(x)+b(x)) = α·a(x)+α·b(x)
(B6) Multiplication is distributive relatively to addition of scalars.
∀a(x)∈V, ∀real α,β: (α+β)·a(x) = α·a(x)+β·a(x)

Let's define a scalar product of two polynomials as an integral of their algebraic product on a segment [0,1].
To differentiate a scalar product of two polynomials from their algebraic product under integration we will use notation [a(x)·b(x)] for a scalar product.
[a(x)·b(x)] = ∫_[0,1] a(x)·b(x) dx

This definition of a scalar product satisfies all the axioms we set for a scalar product in an abstract vector space.
(1) For any polynomial a(x) from V, which is not a null-polynomial, its scalar product with itself is a positive real number
∀a(x)∈V, a(x)≠0(x):
∫_[0,1] a(x)·a(x) dx > 0
(2) For null-polynomial 0(x) its scalar product with itself is equal to zero
∫_[0,1] 0(x)·0(x) dx = 0
(3) Scalar product of any two polynomials a(x) and b(x) is commutative because an algebraic multiplication of polynomials is commutative
∀a(x),b(x)∈V:
∫_[0,1] a(x)·b(x) dx = ∫_[0,1] b(x)·a(x) dx
(4) Scalar product of any two polynomials a(x) and b(x) is proportional to their magnitude
∀a(x),b(x)∈V, for any real γ:
∫_[0,1] (γ·a(x))·b(x) dx =
= γ·∫_[0,1] a(x)·b(x) dx
(5) Scalar product is distributive relatively to addition of polynomials. ∀a(x),b(x),c(x)∈V:
∫_[0,1](a(x)+b(x))·c(x) dx=
= ∫_[0,1] a(x)·c(x) dx + ∫_[0,1] b(x)·c(x) dx

Based on above axioms that are satisfied by polynomials with scalar product defined as we did, we can say that this set is pre-Hilbert space.
The only missing part to be a complete Hilbert space is that this set does not contain limits to certain sequences.
Indeed, we can approximate many smooth non-polynomial functions with sequences of polynomials (recall, for example, Taylor series).

However, the Cauchy-Schwartz-Bunyakovsky inequality was proven for any abstract vector space with scalar product (pre-Hilbert space), and we can apply it to our set of polynomials.
According to this inequality, the following is true for any pair of polynomials:
[a(x)·b(x)]² ≤ [a(x)·a(x)]·[b(x)·b(x)]
or, using our explicit definition of a scalar product,
[∫_[0,1] a(x)·b(x) dx]² ≤
≤ [∫_[0,1] a²(x) dx]·[∫_[0,1] b²(x) dx]

Just out of curiosity, let's see how it looks for a(x)=x^m and b(x)=xⁿ.
In this case
a(x)·b(x) = x^m+n
a²(x) = x^2m
b²(x) = x²ⁿ
Calculating all the scalar products
∫_[0,1] x^(m+n) dx = 1/(m+n+1)
∫_[0,1] x^2m dx = 1/(2m+1)
∫_[0,1] x²ⁿ dx = 1/(2n+1)
Now the Cauchy-Swartz-Bunyakovsky inequality looks like
1/(m+n+1)² ≤ 1/[(2m+1)(2n+1)]

The validity of this inequality is not obvious, so it would be nice to check if it's really true for any m and n.
To check it, let's transform it into an equivalent inequality between denominators with reversed sign of inequality
(m+n+1)² ≥ (2m+1)(2n+1)
Opening all the parenthesis leads us to this equivalent inequality
m²+n²+1+2mn+2m+2n ≥
≥ 4mn+2m+2n+1
After obvious simplification the resulting inequality looks like
m²+n²−2mn ≥ 0
which is always true because the left side equals to (m−n)².
All transformations were invariant and reversible, which proves the original inequality.


Example 2

Elements of our new vector space are infinite sequences of real numbers {x_n} (n changes from 1 to ∞) for which series Σ_n∈[1,∞) x_n² converges to some limit.

Addition and multiplication by a scalar are defined as addition and multiplication individual members of the sequences involved.
These operations preserve the convergence of sum of squares of elements.

Scalar product is defined as
{x_n}·{y_n} = Σ_n∈[1,∞) x_n·y_n
In some sense this is an expansion of N-dimensional Euclidean space to infinite number of dimensions, as long as a scalar product is properly defined, which in our case is assured because of convergence of the sum of squares of the elements.
Indeed, this definition makes sense because each member of a sum that defines a scalar product is bounded
|x_n·y_n| ≤ ½(x_n² + y_n²)
and the sum of right side of this inequality for all n∈[1,∞) converges.

This set is Hilbert space (we skip the proof that this space is complete for brevity), its properties are very much the same as properties of N-dimensional Euclidean space.
All the axioms of Hilbert space are satisfied.
As a consequence, the Cauchy-Shwartz-Bunyakovsky inequality is [{x_n}·{y_n}]² ≤ [{x_n}·{x_n}]·[{y_n}·{y_n}]

Problem A

Given a set of all real two-dimensional vectors (a₁,a₂) with standard definitions of addition and multiplication by a scalar (real number)
(a₁,a₂) + (b₁,b₂) = (a₁+b₁,a₂+b₂)
λ·(a₁,a₂) = (λ·a₁,λ·a₂)
So, it's a linear vector space.

The scalar product we will define in a non-standard way:
(a₁,a₂)·(b₁,b₂) =
= a₁·b₁ + 2·a₁·b₂ + 2·a₂·b₁ + a₂·b₂

Is this vector space a Hilbert space?

Hint A
Check if a scalar product of some vector by itself is zero, while the vector is not a null-vector.

Solution A
Let's examine all vectors that have the second component equal to 1 and find the first component x, which breaks the rule of scalar product of a vector with itself to be positive, unless the vector is a null-vector
(x,1)·(x,1) = 0

According to our non-standard definition of a scalar product, this means the following for a₁=b₁=x and a₂=b₂=1
x·x + 2·x·1 + 2·1·x + 1·1 = 0
x² + 4·x + 1 = 0
x₁ = −2 + √3
x₂ = −2 − √3
So, both vectors (x₁,1) and (x₂,1) have the property that the scalar product of a vector by itself gives zero, while the vectors themselves are not null-vectors.
Indeed, for vector (x₁,1) this scalar product with itself is
(x₁,1)·(x₁,1) = (−2+√3,1)·(−2+√3,1) =
= (4−4√3+3)+4·(−2+√3)+1 = 0
Therefore, thus defined scalar product does not satisfy the axiom for a scalar product in Hilbert space, and our space is not Hilbert's.

Problem B

Prove the parallelogram law in Hilbert space V
∀ a,b∈V:
||a−b||² + ||a+b||² = 2||a||² + 2||b||²

Note B
For vectors in two-dimensional Euclidean space this statement geometrically mean that sum of squares of two diagonals in a parallelogram equals to sum of squares of all its sides.
The parallelogram law can be proven geometrically in this case, using, for example, the Theorem of Cosines.

Hint B
The definition of a norm or magnitude of a vector x in Hilbert space is
||x|| = √(x·x)
Using this, all you need to prove the parallelogram law is to open parenthesis is the magnitudes of a−b and a+b.

Vectors+ 08 Hilbert Space: UNIZOR.COM - Math+ & Problems - Vectors

Notes to a video lecture on http://www.unizor.com

Vectors+ 08 - Hilbert Space

Let's continue building our abstract theory of vector spaces introduced in the previous lecture Vectors 07 of this Vectors chapter of this course Math+ & Problems on UNIZOR.COM.

Our next addition is a scalar product of two elements of an abstract vector space.
In case of N-dimensional Euclidian space with two vectors
R(R₁,R₂,...,R_N) and
S(S₁,S₂,...,S_N)
we defined scalar product as
R·S = R₁·S₁+R₂·S₂+...+R_N·S_N

Thus defined, the scalar product had certain properties and characteristics that we have proven based on this definition.
In case of abstract vectors spaces, the scalar product is not explicitly defined, but, instead, defined as any function of two vectors from our vector space that satisfies certain axioms that resemble the properties and characteristics of a scalar product of two vectors in N-dimensional Euclidian space.

Let's assume, we have a vector space V and a scalar space S associated with it - a set of all real numbers in our case.
All axioms needed for V and S to be a vector space were described in the previous lecture mentioned above.
Now we assume that for any two vectors a and b from V there exists a real number called their scalar product denoted a·b that satisfies the following set of axioms.

(1) For any vector a from V, which is not a null-vector, its scalar product with itself is a positive real number
∀a∈V, a≠0: a·a > 0
(2) For null-vector 0 its scalar product with itself is equal to zero
a=0: a·a = 0
(3) Scalar product of any two vectors a and b is commutative
∀a,b∈V: a·b = b·a
(4) Scalar product of any two vectors a and b is proportional to their magnitude
∀a,b∈V, ∀γ∈S: (γ·a)·b = γ·(a·b)
(5) Scalar product is distributive relatively to addition of vectors. ∀a,b,c∈V: (a+b)·c = a·c+b·c

As before, a square root from a scalar product of a vector by itself will be called magnitude or length, or norm of this vector
||a|| = √(a·a)

Using the above defined scalar product, we can define a distance between vectors a and b as an absolute value of a magnitude of vector a+(−b), which we will write as a−b.
||a−b|| = √(a−b)·(a−b)

A vector space with a scalar product defined above is called a pre-Hilbert space.

To be a Hilbert space, we need one more condition - the completeness of vector space, which means that every converging in a certain way sequence of vectors has a vector that is the limit of this sequence within the same vector space.
More rigorously, the convergence is defined in terms of Cauchy criterion. It states that a sequence of vectors {a_i} converges if
for any ε>0 there is a natural number N such that the distance between a_m and a_n is less than ε for any m,n ≥ N
∀ε>0 ∃N: m,n > N => ||a_m−a_n||<ε


Problem A
Prove that a scalar product of null-vector with any other is zero.

Proof A
∀a∈V: 0·a = (0·a)·a = 0·(a·a) = 0
End of proof.


Problem B
Prove that a scalar product changes the sign, if one of its components is replaced with its opposite.

Proof B
[Problem C from the previous lecture Vectors 07 stated that −a=−1·a]
∀a,b∈V: (−a)·b = (−1·a)·b = −1·(a·b)
End of proof.


Problem C
Prove the Cauchy-Schwartz-Bunyakovsky inequality
∀a,b∈V: (a·b)² ≤ (a·a)·(b·b).

Proof C
If either a or b equals to null-vector, we, obviously, get zero on both sides of inequality, which satisfies the sign ≤.
Assume, both vectors are not-null.
Consider any non-zero γ and non-negative scalar product of a+γb by itself
0 ≤ (a+γ·b)·(a+γ·b)
Use commutative and distributive properties to open all parenthesis
0 ≤ a·a+2γ·a·b+γ²·b·b
Set γ=−(a·b)/[(b·b)]
With this the inequality above takes form
0 ≤ a·a−2·(a·b)²/(b·b)+(a·b)²/(b·b)
Multiplying this inequality by a positive b·b, we obtain
0 ≤ (a·a)·(b·b)−2·(a·b)²+(a·b)²
which transforms into
(a·b)² ≤ (a·a)·(b·b)
End of proof.

Vectors+ 07 Abstract Vector Space: UNIZOR.COM - Math+ & Problems - Vectors

Notes to a video lecture on http://www.unizor.com

Vectors+ 07 - Abstract Vector Space

This lecture represents a typical mathematical approach to move from a relatively simple and concrete object to some abstraction that allows to transfer properties of the original simple object to many others without proving these properties for each particular case.

Let's start with a simple prototype - a two-dimensional Euclidian plane with coordinates and vectors we studied before. Coordinates are real numbers, vectors can be added to each other and multiplied by a real numbers resulting in some other vector.

Our first step towards abstraction was to move to N-dimensional space, which retains most of the properties of two-dimensional one. However, this is not the exact step towards abstraction, because we still considered a concrete object - an N-dimensional space and demonstrated its properties.

The real abstraction is to choose any object that satisfies certain axioms and derive the properties from these axioms without referring to any concrete object.
Here is what can be done in this direction.

First of all, we introduce an abstract set V, whose elements will be called vectors. It's modelled after an N-dimensional Euclidean space as a prototype, but, in theory, it can be any set satisfying the rules below.

We also consider a set S, whose elements will be called scalars. Its prototype is a set of all real numbers used for multiplication by vectors to change their magnitude, and for our purposes, to concentrate attention on vectors, we will consider this set of scalars to be exactly that, a set of all real numbers. However, it can be some other set, like all complex numbers.

We postulate that these sets satisfy the following rules.

(A) The operation of addition is defined for each pair of vectors in V that establishes a correspondence of this pair of vectors to a new vector called their sum. What's important, we don't define this operation, we don't establish any process it should follow. The only requirement is that this operation must satisfy the following axioms.
(A1) Addition of any two vectors a and b is commutative
∀a,b∈V: a + b = b + a
(A2) Addition of any three vectors a, b and c is associative
∀a,b,c∈V: (a + b) + c = a + (b + c)
(A3) There is one vector called null-vector, denoted 0, with a property of not changing the value of any other vector a if added to it
∀a∈V: a + 0 = a
(A4) For any vector a there is another vector called its opposite, denoted as −a, such that the sum of a vector and its opposite equals to null-vector
∀a∈V ∃(−a)∈V: a + (−a) = 0
Note: in many cases an expression a+(−b) will be shortened to a−b. Sign '−' here does not mean a new operation of subtraction, but just an indication of addition with an opposite element.

(B) The operation of multiplication of vector by scalar is defined for each vector in V and each scalar in S, taken in any order. This operation establishes a correspondence of this vector and this scalar to a new vector called the product of a vector and a scalar.
This operation must satisfy the following axioms.
(B1) Multiplication of any scalar α by any vector a is commutative
∀a∈V, ∀α∈S: α·a = a·α
(B2) Multiplication of any two scalars α and β by any vector a is associative
∀a∈V, ∀α,β∈S: (α·β)·a = α·(β·a)
(B3) Multiplication of any vector by scalar 0 results in null-vector
∀a∈V: 0·a = 0
(B4) Multiplication of any vector by scalar 1 does not change the value of this vector a
∀a∈V: 1·a = a
(B5) Multiplication is distributive relatively to addition of vectors. ∀a,b∈V, ∀α∈S: α·(a+b) = α·a+α·b
(B6) Multiplication is distributive relatively to addition of scalars.
∀a∈V, ∀α,β∈S: (α+β)·a = α·a+β·a


Problem A
Prove that there must be only one null-vector in vector space V.

Proof A
Assume there are two null-vectors 0₁ and 0₂.
Since 0₂ is null-vector, its addition to 0₁ does not change 0₁.
0₁ + 0₂ = 0₁
Since 0₁ is null-vector, its addition to 0₂ does not change 0₂.
0₂ + 0₁ = 0₂
Since addition is commutative, left sides in the two equations above are equal.
0₁ + 0₂ = 0₂ + 0₁
Therefore, the right sides are equal:
0₁ = 0₂
End of proof.


Problem B
Prove that for any vector there must be only one opposite to it vector in vector space V.

Proof B
Assume that for some vector a there are two opposite vectors (−a)₁ and (−a)₂.
Since (−a)₁ is an opposite to vector a, its addition to a results in null-vector.
Therefore,
((−a)₁+a) + (−a)₂ = 0 + (−a)₂ = (−a)₂
Since (−a)₂ is an opposite to vector a, its addition to a results in null-vector.
Therefore,
(−a)₁ + (a+(−a)₂) = (−a)₂ + 0 = (−a)₂
Since addition is associative, left sides in the two equations above are equal.
Therefore, the right sides are equal:
(−a)₁ = (−a)₂
End of proof.


Problem C

Prove that for any vector its product with scalar −1 results in an opposite vector.

Proof C
Since vector multiplication by scalar is distributive,
(1+(−1))·a = 1·a + (−1)·a = a + (−1)·a
On the other hand, the same initial statement can be transformed differently
(1+(−1))·a = 0·a = 0
Since left sides in the two equations above are equal, right sides are equal as well
a + (−1)·a = 0
Therefore, (−1)·a satisfies the definition of a vector opposite to a. But, as has been proven in Problem B, there must be only one vector opposite to a, which we denoted as (−a).
Hence, (−1)·a = (−a)
End of proof.

Vectors+ 06 More N-vectors: UNIZOR.COM - Math+ &Problems - Vectors

Notes to a video lecture on http://www.unizor.com

Vectors+ 06
Cauchy-Schwarz-Bunyakovsky Inequality

This inequality is also known as Cauchy-Bunyakovsky or Cauchy-Schwarz inequality.
We will consider this inequality only for a case of N-dimensional real vectors, where it states that scalar product of two vectors is less or equal to a product of their magnitudes
R·S ≤ ||R||·||S||
In terms of components,
R (R₁,R₂,...,R_N)
S (S₁,S₂,...,S_N)
R·S = R₁·S₁+R₂·S₂+...+R_N·S_N
||R|| = √R₁²+R₂²+...+R_N²
||S|| = √S₁²+S₂²+...+S_N²
To prove the above inequality, it is sufficient to prove it for squares of both left and right parts, since the right part is always positive, while the left (smaller or equal) can take negative values as well. This allows to get rid of the square roots.

Let's prove that
(R·S)² ≤ (||R||·||S||)²
In coordinate form we have to prove that
(R₁·S₁+R₂·S₂+...+R_N·S_N)² ≤
(R₁²+R₂²+...+R_N²)·(S₁²+S₂²+...+S_N²)

Proof
Consider a quadratic polynomial for an unknown variable x
P(x) = (R₁·x+S₁)²+(R₂·x+S₂)²+...
...+(R_N·x+S_N)²
This quadratic polynomial is, obviously, non-negative, as it contains only non-negative components added together.
This same polynomial can be regrouped by opening parenthesis and presented as
P(x) = (R₁²+R₂²+...+R_N²)·x² +
+ 2·(R₁·S₁+R₂·S₂+...+R_N·S_N)·x +
+ (S₁²+S₂²+...+S_N²) =
= Ax² + Bx + C
where
A = ||R||²
B = 2·R·S
C = ||S||²
In order to have a non-negative polynomial, its necessary and sufficient to have its discriminant to be non-positive.
Therefore,
B²−4·A·C ≤ 0
(2·R·S)² − 4·||R||²·||S||² ≤ 0
(R·S)² ≤ ||R||²·||S||²
End of Proof.


Consequences

1. Triangle inequality
Triangle inequality deals with a magnitude of three vectors R, S and their sum R+S that completes a triangle in N-dimensional space.
In particular, the Triangle inequality states that in any triangle sum of two sides as greater or equal than the third one.
As we know,
||R+S||² = (R+S)·(R+S) =
= R·R + S·S + 2·R·S ≤
[applying the Cauchy-Schwartz inequality to the third term]
≤ ||R||² + ||S||² +2·||R||·||S|| =
= (||R||+||S||)²
Therefore, ||R+S|| ≤ ||R||+||S|| which is the Triangle inequality.

2. The cosine between vectors
We have defined a cosine of the angle between vectors R and S in N-dimensional space as
cos(φ) = R·S/(||R||·||S||).
Because of the Cauchy-Schwartz inequality the above expression is always between −1 and 1 and, therefore, can be a cosine of some angle. So, our definition of an angle in N-dimensional space makes total sense.

Vectors+ 05 More about N-vectors: UNIZOR.COM - Math+ & Problems - Vectors

Notes to a video lecture on http://www.unizor.com

Vectors+ 05
More N-dimensional Vectors

Theory

Let's summarize the characteristics of N-dimensional vectors.
First of all an N-dimensional vector is an ordered set of N real numbers (R₁,R₁,...,R_N) that we will use as a single object R.
Real numbers R_n where n∈[1,N] will be referred as components of R.
Unless specifically noted, we will assume that the word vector in this lecture means N-dimensional vector.

1. Addition
Any two vectors can be added together giving another vector according to a simple rule
R (R₁,R₂,...,R_N) + S (S₁,S₂,...,S_N) =
= T (R₁+S₁,R₂+S₂...,R_N+S_N,)
Addition of vectors is, obviously, commutative. It follows from the commutative property of the addition of real numbers.
R + S = S + R because
R_n + S_n = S_n + R_n
for n∈[1,N]
Also, addition of vectors is associative. It follows from the associative property of the addition of real numbers.
(R + S) + T = R + (S + T) because
(R_n + S_n) + T_n = R_n + (S_n + T_n)
for n∈[1,N]

2. Null-vector
Vector 0 (0,0,...,0) plays the same role for vector addition as real number 0 with other real numbers, namely, adding null-vector to any vector does not change that vector
R + 0 = 0 + R = R
which directly follows from the rules of addition of vectors and properties of real number 0.
For any non-null vector
R (R₁,R₂,...,R_N)
there is an opposite vector with the signs of all components reversed
−R (−R₁,−R₂,...,−R_N)
that, if added to R, results in a null-vector sum
R + (−R) = 0

3. Multiplication by a scalar
Any vector can be multiplied by a scalar, which is defined as all components of this vector to be multiplied by this scalar.
R (R₁,R₂,...,R_N)·f = T (R₁·f,R₂·f...,R_N·f)
Analogously, multiplication of a scalar by a vector is
f·R (R₁,R₂,...,R_N) = T (f·R₁,f·R₂...,f·R_N)
which, obviously, is the same as multiplication of a vector by a constant because of commutative properties of multiplication of real numbers.

4. Scalar (dot) product of two vectors
Any two vectors can form a scalar product giving a scalar according to a simple rule
R (R₁,R₂,...,R_N) · S (S₁,S₂,...,S_N) =
= R₁·S₁+R₂·S₂+...+R_N·S_N
Obviously, it's commutative because of commutative properties of multiplication of real numbers.

5. Magnitude of a vector
For any vector
R (R₁,R₂,...,R_N)
we define its magnitude (length) ||R|| in a way to be similar to 2- and 3- dimensional cases:
||R|| = √(R·R) = √R₁²+R₂²+...+R_N².

6. Orthogonality
We will call two vectors orthogonal to each other, if their scalar product is zero.

7. Basis
The following set of N vectors
i⁽¹⁾ (1,0,...,0)
i⁽²⁾ (0,1,...,0)
...
i^(N) (0,0,...,1)
consists of mutually orthogonal vectors of unit length.
This directly follows from the definitions of scalar product and magnitude of vectors.
Indeed,
i^(m)·i⁽ⁿ⁾ = 0 for m,n∈[1,N] and m≠n
√(i^(m)·i^(m)) = 1 for each m∈[1,N]
They are called unit vectors.
These unit vectors form orthogonal basis because any vector
R (R₁,R₂,...,R_N)
can be represented as the linear combination of these unit vectors
R = R₁·i⁽¹⁾ + R₂·i⁽²⁾ +...+ R_N·i^(N)

8. Angle
For two-dimensional vectors
U ·V = u·v·cos(φ)
where u is the magnitude of vector U,
v is the magnitude of vector V and
φ is the angle between them in any direction since cos(φ)=cos(−φ).
Accordingly, as explained in the previous lecture, we define an angle between two N-dimensional vectors as
cos(φ) = (U·V)/(||U||·||V||)


Problem A

Prove that multiplication of a vector by a scalar is distributive.
That is,
(a) f·(R + S) = f·R + f·S
(b) (f + g)·R = f·R + g·R


Problem B

Prove that scalar product of two vectors is distributive.
That is,
R·(S + T) = R·S + R·T


Problem C

Two three-dimensional vectors are given
OA (2,3,4)
OB (−2,1,3)
Imagine a triangle in three-dimensional space ΔOAB formed by these two vectors and their difference:
OB−OA = AB
Determine the lengths of three sides of this triangle, cosines of its angles and check the Theorem of Cosine in it.

Solution

According to definition of a difference of two vectors, vector AB is (−2−2,1−3,3−4)=(−4,−2,−1).

Now we can calculate all magnitudes.
OA = √2²+3²+4² = √29
OB = √(−2)²+1²+3² = √14
AB = √(−4)²+(−2)²+(−1)² = √21

Scalar products are
OA·OB =
= 2·(−2)+3·1+4·3 = 11
OA·AB =
= 2·(−4)+3·(−2)+4·(−1) = −18
OB·AB =
= (−2)·(−4)+1·(−2)+3·(−1) = 3

Before calculating the cosines of angle of ΔOAB let's draw this triangle in the plane it belongs to.
We have calculated its sides, so it should look like this

Angle ∠AOB is an angle between vectors OA and OB.
cos(∠AOB) = (OA·OB)/(OA·OB) =
= 11/√(29·14)

Angle ∠OAB is NOT an angle between vectors OA and AB. It is, actually, an angle between vectors AO (opposite to OA, the vector of the same length but opposite direction) and AB.
The vector AO, as opposite to OA(2,3,4), is a set of real values (−2,−3,−4) and, therefore, its scalar product with AB is
AO·AB =
= (−2)·(−4)+(−3)·(−2)+(−)4·(−1) =
= 18
Therefore,
cos(∠OAB) = (AO·AB)/(AO·OB) =
= 18/√(29·21) = 18/√(29·21)

Angle ∠OBA is vertical (and, therefore, equal) to an angle between vectors OB and AB.
cos(∠OBA) = (OB·AB)/(OB·AB) =
= 3/√(14·21)

Let's check the Theorem of Cosines.

AB²=OA²+OB²−OA·AB·cos(∠AOB) =
= 29 + 14 − 2·√(29·14)·11/√(29·14) =
= 29 + 14 − 22 = 21 ∴ CORRECT

OA²=OB²+AB²−OB·AB·cos(∠OBA) =
= 14 + 21 − 2·√(14·21)·3/√(14·21) =
= 14 + 21 − 6 = 29 ∴ CORRECT

OB²=OA²+AB²−OA·AB·cos(∠OAB) =
= 29 + 21 − 2·√(29·21)·18/√(29·21) =
= 29 + 21 − 36 = 14 ∴ CORRECT

Vectors+ 04 - N-dimensional Vectors: UNIZOR.COM - Math+ & Problems - Vec...

Notes to a video lecture on http://www.unizor.com

Vectors+ 04
N-dimensional Vectors

Theory

1. A single real number R can be represented as a point A on a straight coordinate line having some fixed point O called the origin, unit of measurement and a particular direction from point O chosen as positive.
In this case point A is defined so that its distance from the origin O equals to |R| in chosen units of measurement and the sign of R corresponding to a defined positive or negative direction from the origin.
At the same time this number R can be represented as a vector that stretches along the coordinate line from one point to another having the magnitude of absolute value |R| and the direction corresponding to a defined positive or negative direction from the origin.
So, a single real number R is represented as a point A on a coordinate line or as a vector R on this line.
All these representations of a single real number we call one-dimensional.
In the vector representation real number R=1 can be represented by a vector from the origin of coordinates O to point on a distance of 1 unit of measurement in a positive direction. We will call this vector a unit vector i.
If we stretch vector i by a factor of R, taking in consideration the correspondence between the sign of R and direction on a line, we will reach point A.
That's why we can state
R = R·i

2. A pair of two real numbers (R₁,R₂) can be represented as a point A on a coordinate plane with some fixed point O called the origin of coordinates, two perpendicular lines with chosen positive direction on each going through the origin O called abscissa and ordinate with some unit of measurement along these lines.
In this case point A is defined so that
(a) its distance from the ordinate along a line parallel to the abscissa equals to |R₁| in chosen units of measurement and the sign of R₁ corresponding to a defined positive or negative direction along the abscissa;
(b) its distance from the abscissa along a line parallel to the ordinate equals to |R₂| in chosen units of measurement and the sign of R₂ corresponding to a defined positive or negative direction along the ordinate.
At the same time this pair of numbers (R₁,R₂) can be represented as a vector that stretches from the origin of coordinate to point A defined above or any other vector on a plane with the same magnitude and direction.
So, a pair of real numbers (R₁,R₂) is represented as a point A on a coordinate plane or as a vector R from the origin O to point A, or any other vector of the same magnitude and direction.
All these representations of a pair of real numbers we call two-dimensional.

In the vector representation a pair of real numbers (R₁=1,R₂=0) is represented by vector from the origin along the abscissa to point I₁ on a distance of 1 unit of measurement in a positive direction. We will call this vector a abscissa unit vector i.
Similarly, the vector representation a pair of real numbers (R₁=0,R₂=1) is represented by vector from the origin along the ordinate to point I₂ on a distance of 1 unit of measurement in a positive direction. We will call this vector an ordinate unit vector j.
If we stretch vector i by a factor of R₁, taking in consideration the correspondence between the sign of R₁ and direction on the abscissa, and similarly stretch vector j by a factor of R₂ and add these two vectors by the rules of addition of vectors, we will reach point A.
That's why we can state
R = R₁·i + R₂·j
which is a representation of vector R through unit vectors i and j and coordinates R₁ and R₂.

Using the above representation of a vector, it's easy to derive a formula for a scalar product of two two-dimensional vectors.
R = R₁·i + R₂·j
S = S₁·i + S₂·j
R·S = R₁·S₁·(i·i) + R₁·S₂·(i·j) +
+ R₂·S₁·(j·i) + R₂·S₂·(j·j)

According to definition of a scalar product, (i·i)=1, (i·j)=0, (j·i)=0, (j·j)=1
Therefore, R·S = R₁·S₁ + R₂·S₂

Perpendicularity of vectors R and S can be checked by examining the coordinate expression for their scalar product R₁·S₁+R₂·S₂. If it's equal to zero, the vectors are perpendicular to each other.

A cosine of an angle φ between two two-dimensional vectors can be determined as
cos(φ) = (R·S) / (|R|·|S|)
where in numerator we use scalar product of two vectors and in denominator - a product of their magnitudes.

3. Exactly as in a previous two-dimensional case, we can represent a triplet of real numbers (R₁,R₂,R₃) as a point A in the three-dimensional space, a vector R from the origin of coordinates to that point or as a vector sum of three mutually perpendicular vectors, each positioned along one of the coordinate axes
R = R₁·i + R₂·j + R₃·k
where i, j and k are unit vectors along three mutually perpendicular axes of coordinates.

Using the above representation of a vector, it's easy to derive a formula for a scalar product of two three-dimensional vectors.
R = R₁·i + R₂·j + R₃·k
S = S₁·i + S₂·j + S₃·k
R·S = R₁·S₁ + R₂·S₂ + R₃·S₃

Perpendicularity of vectors R and S can be checked by examining the coordinate expression for their scalar product R₁·S₁+R₂·S₂+R₃·S₃. If it's equal to zero, the vectors are perpendicular to each other.

A cosine of an angle φ between two two-dimensional vectors can be determined as
cos(φ) = (R·S) / (|R|·|S|)
where in numerator we use scalar product of two vectors and in denominator - a product of their magnitudes.

4. Now we will generalize the same concept to N-dimensional case.
An ordered set of N real numbers (R₁,R₂,...,R_N) we will call an N-dimensional vector, which can be interpreted as a point in N-dimensional coordinate space.
An important operations on vectors known from 2- and 3-dimensional cases can be easily expanded to N-dimensional case:

(a) Addition of two N-dimensional vectors resulting in a new N-dimensional vector:
R⁽¹⁾ + R⁽²⁾ = R⁽²⁾ + R⁽¹⁾ = R
(R₁⁽¹⁾,...,R_N⁽¹⁾) + (R₁⁽²⁾,...,R_N⁽²⁾) =
= (R₁⁽¹⁾+R₁⁽²⁾,...,R_N⁽¹⁾+R_N⁽²⁾)

(b) Multiplication of an N-dimensional vector by a real number q resulting in a new N-dimensional vector:
q·R = R·q = S
q·(R₁,...,R_N) = (q·R₁,...,q·R_N)

(c) Scalar (dot) product of two N-dimensional vectors resulting in a scalar:
R⁽¹⁾ · R⁽²⁾ = R⁽²⁾ · R⁽¹⁾ = C
(R₁⁽¹⁾,...,R_N⁽¹⁾) · (R₁⁽²⁾,...,R_N⁽²⁾) =
= R₁⁽¹⁾·R₁⁽²⁾+...+R_N⁽¹⁾·R_N⁽²⁾ = C

(d) N-dimensional Angle
If a scalar product of two N-dimensional vectors R⁽¹⁾ and R⁽²⁾ equals to zero, they are called perpendicular to each other.
In general, an angle φ between these two N-dimensional vectors can be defined as
cos(φ) = (R⁽¹⁾·R⁽²⁾) / (|R⁽¹⁾|·|R⁽²⁾|)
where in numerator we use scalar product of two vectors and in denominator - a product of their magnitudes.

(e) Linear dependency
Certain number K of N-dimensional vectors R⁽¹⁾,...,R^(K) is called linearly dependent if there are K multipliers q₁,...,q_K, not all of which are equal to zero, such that
q₁·R⁽¹⁾+...+q_K·R^(K) = 0
where 0 is a null-vector - an N-dimensional vector with all components zero.
The negation of linear dependency is linear independency.


Problem A

Prove that in N-dimensional space exists a set of N linearly independent vectors.

Solution A
We can prove the existence by suggesting a concrete set of vectors that satisfies the requirements of a problem.
Consider these N vectors in N-dimensional space
i⁽¹⁾ = (1,0,...,0),
i⁽²⁾ = (0,1,...,0),
...
i^(N) = (0,0,...,1)
They are linearly independent.
Indeed, assume there exist multipliers q₁,...,q_K, not all of which are equal to zero, such that
q₁·i⁽¹⁾+...+q_N·i^(N) = 0
But expression on the left is vector
S = (q₁,...,q_N)
So, if it's equal to null-vector, all components are zero, which contradicts the assumption about linear dependency of these vectors.
Hence, these N vectors are linearly independent.


Problem B

Represent any vector in N-dimensional space as a linear combination of the set of linearly independent N vectors presented in the solution of the previous problem.

Solution B
Take any N-dimensional vector R = (R₁,...,R_N)
and set of linearly independent N-dimensional vectors
i⁽¹⁾ = (1,0,...,0),
i⁽²⁾ = (0,1,...,0),
...
i^(N) = (0,0,...,1)
It is obvious that
R = R₁·i⁽¹⁾+R₂·i⁽²⁾+...+R_N·i^(N)


Problem C

R⁽¹⁾, R⁽²⁾,... ,R^(N) are N linearly independent vectors in N-dimensional vector space, where for each k∈[1,N] vector R^(k) is a set of real numbers (R₁^(k),R₂^(k),...,R_N^(k)).
Represent any vector V(V₁,V₂,...,V_N) in this N-dimensional space as a linear combination of these vectors R^(k), where k∈[1,N].

Solution C

We are looking for a representation that in vector form looks like
V = x₁·R⁽¹⁾ + x₂·R⁽²⁾ +...+ x_N·R^(N)
where for k∈[1,N] all x_k are unknown real numbers.
Let's state this equation in coordinate form
V₁ = R₁⁽¹⁾·x₁+R₁⁽²⁾·x₂+...+R₁^(N)·x_N
V₂ = R₂⁽¹⁾·x₁+R₂⁽²⁾·x₂+...+R₂^(N)·x_N
...
V_N = R_N⁽¹⁾·x₁+R_N⁽²⁾·x₂+...+R_N^(N)·x_N
This system of linear equations has unique solution if the N⨯N matrix of coefficients R_i^(j) has a non-zero determinant. This was explained in the Math 4 Teens course on UNIZOR.COM (see Matrices part of the course, chapters Matrix Determinant and Matrix Solution).
Exactly the same criteria of non-zero determinant is a necessary and sufficient condition for a set of vectors to be linearly independent.
Since the linear independence of vectors R⁽¹⁾, R⁽²⁾,... ,R^(N) is given, the system of linear equations has a unique solution for unknown coefficients x_i.

Answer C
From the coordinates of all N vectors R^(j) (j∈[1,N]) and coordinates V_i (i∈[1,N]) of vector V construct a system of N linear equations.
It's solutions gives the coefficients x_j we are looking for.

Definition

N linearly independent vectors in N-dimensional vector space are called basis, if any vector of this vector space can be represented as a linear combination of these N linearly independent vectors.
From the Problems A, B follows that a set of vectors
i⁽¹⁾ = (1,0,...,0),
i⁽²⁾ = (0,1,...,0),
...
i^(N) = (0,0,...,1)
is a basis.
So is a set of vectors R⁽¹⁾, R⁽²⁾,... ,R^(N) of Problem C.