# Linear Algebra: Matrices, Linear Systems and Eigenvalues > Linear algebra notes covering linear systems, matrices, determinants, vectors, rank, eigenvalues, eigenvectors and diagonalization. - Author: Muhammet Ali Köker - Language: en - Canonical: https://alikoker.com.tr/en/linear-algebra-matrices-linear-systems-eigenvalues - Translation: https://alikoker.com.tr/lineer-cebir-matrisler-lineer-sistemler-ozdegerler - Published: 2013-11-24T18:10:00+03:00 - Modified: 2026-08-09T00:12:00+03:00 - Verified: 2026-08-09T00:12:00+03:00 - Type: article I prepared my linear-algebra notes during undergraduate Computer Engineering courses in the 2013-2014 period. This revision preserves the original sequence from linear systems and matrices through determinants, vectors, rank, eigenvalues, and diagonalization, while reviewing numerical and conceptual distinctions with later sources. ## Unit 1: Linear Equations and Matrices ### Linear equation A linear equation in variables `x1, x2, ..., xn` has the form ```text a1 x1 + a2 x2 + ... + an xn = b. ``` The coefficients and constant term are scalars. Variables appear only to the first power and are not multiplied together. ### System of linear equations A linear system is a collection of linear equations sharing the same unknowns. Depending on consistency and rank, a system can have one solution, no solution, or infinitely many solutions. ### Geometry of a two-variable system In two dimensions, each nondegenerate linear equation represents a line. Two distinct nonparallel lines intersect at one point; parallel distinct lines have no common solution; coincident lines represent infinitely many common solutions. ### Elementary operations and elimination The solution set is preserved by exchanging equations, multiplying an equation by a nonzero scalar, and adding a multiple of one equation to another. These are the algebraic basis of elimination. ### Matrix concept A matrix organizes coefficients in rows and columns. An `m x n` matrix has `m` rows and `n` columns. Matrices provide a compact representation of systems and linear transformations. ### Addition, subtraction, and scalar multiplication Matrices of the same dimensions are added or subtracted element by element. Scalar multiplication multiplies every entry by the same scalar. ### Matrix multiplication If `A` is `m x n` and `B` is `n x p`, then ```text (AB)_ij = Σ_k a_ik b_kj. ``` The product exists only when the inner dimensions match. In general, ```text AB ≠ BA. ``` ### Special matrices Frequently used forms include the zero matrix, identity matrix, diagonal matrix, scalar matrix, upper and lower triangular matrices, symmetric matrix, and transpose. Structural properties can simplify later computations. ## Unit 2: Solving Linear Systems with Matrices ### Matrix representation A linear system can be written as ```text Ax = b. ``` Here `A` is the coefficient matrix, `x` is the unknown vector, and `b` is the right-hand-side vector. ### Augmented matrix The augmented matrix ```text [A | b] ``` stores the coefficients and right-hand side in one array and is convenient for row reduction. ### Gaussian elimination Gaussian elimination uses elementary row operations to transform the augmented matrix to row-echelon or upper-triangular form. Back substitution then recovers the unknowns. ### Gauss-Jordan method Gauss-Jordan elimination continues row reduction until pivot columns are reduced further, ideally producing reduced row-echelon form. This makes free variables and consistency conditions directly visible. ### Reading the solution set A contradictory row such as ```text 0 0 ... 0 | c, c ≠ 0 ``` shows inconsistency. If the system is consistent and every variable column contains a pivot, the solution is unique. Free variables imply infinitely many solutions. ### Inverse matrix For a square matrix `A`, an inverse `A^-1` satisfies ```text AA^-1 = A^-1A = I. ``` An inverse exists exactly when `A` is nonsingular. ### Solving with the inverse matrix If `A` is invertible, ```text Ax = b x = A^-1 b. ``` This identity is useful conceptually. In practical numerical computation, explicit inversion is usually not the preferred way to solve a single linear system; factorization or elimination is more efficient. ## Unit 3: Determinants ### Definition of the determinant The determinant maps a square matrix to a scalar. It captures information about invertibility, orientation, and volume scaling. ### Second- and third-order determinants For ```text A = [a b; c d], ``` ```text det(A) = ad - bc. ``` For `3 x 3` matrices, cofactor expansion or Sarrus' rule can be used. Sarrus' rule applies only to `3 x 3` determinants and does not generalize to higher dimensions. ### Properties of determinants Important properties include: - interchanging two rows changes the sign; - multiplying a row by `k` multiplies the determinant by `k`; - adding a multiple of one row to another leaves the determinant unchanged; - a triangular matrix has determinant equal to the product of its diagonal entries; - `det(AB) = det(A)det(B)`; - `det(A^T) = det(A)`. ### Determinants using row operations Row reduction can transform the matrix to triangular form while tracking row exchanges and scalings. This avoids the combinatorial cost of a large cofactor expansion. ### Minor and cofactor The minor `M_ij` is the determinant obtained after deleting row `i` and column `j`. The cofactor is ```text C_ij = (-1)^(i+j) M_ij. ``` ### Cofactor expansion A determinant can be expanded along any row or column: ```text det(A) = Σ_j a_ij C_ij. ``` The value is independent of the chosen expansion row or column. ## Unit 4: Invertibility, Adjugate Matrix, and Cramer's Rule ### Invertibility criterion using the determinant For a square matrix, ```text det(A) ≠ 0 ``` is equivalent to invertibility. A zero determinant means the matrix is singular. ### Adjugate matrix The adjugate is the transpose of the cofactor matrix. For a nonsingular matrix, ```text A^-1 = adj(A) / det(A). ``` This is a closed general formula and is useful for theory and small matrices. For large numerical problems, cofactor-based inversion is normally replaced by factorization and linear-system solvers. ### Cramer's rule For a nonsingular square system, ```text x_i = det(A_i)/det(A), ``` where `A_i` is obtained by replacing column `i` of `A` with `b`. The rule is mathematically elegant but not computationally efficient for large systems. ## Unit 5: Vectors ### Scalar and vector quantities A scalar has magnitude only. A vector has magnitude and direction and can be represented by ordered components in a chosen coordinate system. ### Vector operations Vectors can be added, subtracted, and multiplied by scalars componentwise. These operations satisfy the vector-space laws. ### Components and magnitude For ```text v = (v1, v2, ..., vn), ``` the Euclidean magnitude is ```text ||v|| = sqrt(v1^2 + v2^2 + ... + vn^2). ``` ### Dot product The dot product is ```text u·v = Σ_i u_i v_i. ``` In Euclidean space, ```text u·v = ||u|| ||v|| cos θ. ``` For nonzero vectors, a zero dot product means orthogonality. ### Cross product In three dimensions, ```text u x v ``` is perpendicular to both `u` and `v` and has magnitude ```text ||u x v|| = ||u|| ||v|| sin θ. ``` If two nonzero vectors have zero cross product, they are parallel. ### Scalar triple product The scalar triple product ```text u · (v x w) ``` represents the signed volume of the parallelepiped generated by the three vectors. A zero value indicates coplanarity. ### Linear combination A vector `v` is a linear combination of `v1,...,vk` if ```text v = c1 v1 + ... + ck vk. ``` The set of all such combinations is their span. ### Linear dependence and independence Vectors are linearly independent if ```text c1 v1 + ... + ck vk = 0 ``` implies all coefficients are zero. Otherwise they are linearly dependent. ## Unit 6: Rank and the Structure of Solutions ### Rank of a matrix The rank is the dimension of the row space, equivalently the column space. Computationally it equals the number of pivots in a row-echelon form. ### Rank and linear independence Rank measures how many independent row or column directions the matrix contains. A set of columns is independent exactly when those columns contribute separate pivot directions. ### Linear systems and rank For ```text Ax = b, ``` the system is consistent exactly when ```text rank(A) = rank([A|b]). ``` If this common rank equals the number of unknowns, the solution is unique. If it is smaller, consistent systems have free variables and infinitely many solutions. ### Homogeneous systems For ```text Ax = 0, ``` the zero vector is always a solution. A nontrivial solution exists when the null space has positive dimension; for a square matrix this occurs when `det(A)=0`. ### Comparing solution methods Gaussian elimination is the general direct framework. Gauss-Jordan exposes reduced form and free variables. Inverse and Cramer formulas are useful conceptually and for small problems, but they are not preferred for large numerical systems. ## Unit 7: Eigenvalues and Eigenvectors ### Basic concept A nonzero vector `v` is an eigenvector of `A` if ```text Av = λv. ``` The scalar `λ` is the corresponding eigenvalue. The transformation changes the magnitude and possibly the sign/direction along that eigenvector without changing its one-dimensional span. ### Characteristic equation Rearranging gives ```text (A - λI)v = 0. ``` A nonzero solution exists only if ```text det(A - λI) = 0. ``` This is the characteristic equation. ### Computation procedure First solve the characteristic equation for eigenvalues. For each eigenvalue, solve ```text (A - λI)v = 0 ``` to obtain the corresponding eigenspace. ### Properties of eigenvalues The sum of eigenvalues counted with algebraic multiplicity equals the trace, and their product equals the determinant. For triangular matrices, the eigenvalues are the diagonal entries. ### Basis and dimension A basis is a linearly independent spanning set. Every basis of a finite-dimensional vector space contains the same number of vectors; that number is the dimension. ### Diagonalization A matrix is diagonalizable if there is an invertible matrix `P` such that ```text A = P D P^-1, ``` with diagonal `D`. The columns of `P` are independent eigenvectors. Having `n` linearly independent eigenvectors is the key requirement for an `n x n` matrix. ### Orthogonality and orthonormality Vectors are orthogonal when their dot product is zero. They are orthonormal when they are also unit vectors. Orthonormal bases simplify coordinates, projections, and numerical computation. ### Gram-Schmidt method Gram-Schmidt converts an independent set into an orthogonal or orthonormal set spanning the same subspace. Each new vector has its projections onto the previously constructed directions removed. ### Diagonalization of symmetric matrices A real symmetric matrix has real eigenvalues and can be diagonalized by an orthogonal matrix: ```text A = Q D Q^T. ``` This spectral structure is particularly important in numerical analysis, optimization, and quadratic forms. ## General Conceptual Framework The course connects systems, matrices, vector spaces, and transformations: ```text Linear equations ↓ Matrix representation ↓ Row reduction / rank ↓ Vector spaces and independence ↓ Linear transformations ↓ Eigenvalues and invariant directions ``` Determinants provide compact structural tests, but row reduction and factorization are the practical tools for systems. Rank explains consistency and degrees of freedom. Eigenvectors describe directions preserved by a linear transformation and lead to diagonal representations when enough independent eigenvectors exist. ## Conceptual Distinctions **Matrix multiplication ≠ elementwise multiplication.** The row-by-column rule defines the ordinary product, and multiplication is generally not commutative. **Determinant ≠ rank.** The determinant is a scalar defined for square matrices; rank is defined for rectangular matrices and measures independent directions. **Sarrus' rule ≠ a general determinant algorithm.** It applies only to `3 x 3` matrices. **Inverse formula ≠ preferred large-scale solver.** The adjugate formula is mathematically valid, but direct factorization or elimination is usually preferable numerically. **Dot product ≠ cross product.** A zero dot product indicates orthogonality. For two nonzero vectors in three dimensions, a zero cross product indicates parallelism. **Linear dependence ≠ geometric equality.** Dependence means that at least one vector can be expressed from the others through a nontrivial linear relation. **Rank(A) ≠ rank([A|b]) in an inconsistent system.** Equality is the consistency criterion. **Eigenvalue ≠ eigenvector.** The eigenvalue is a scalar; the eigenvector is a nonzero direction satisfying `Av=λv`. **Algebraic multiplicity ≠ geometric multiplicity.** Repetition in the characteristic polynomial does not automatically provide the same number of independent eigenvectors. **Diagonalizable ≠ invertible.** A matrix may be diagonalizable and singular, or invertible without being diagonalizable. **Orthogonal ≠ orthonormal.** Orthonormal vectors are orthogonal and also normalized to unit length. ## References - Ahmet Yesevi Üniversitesi Bilgisayar Mühendisliği Bölümü. *Lineer Cebir* (TBIL203) ders materyalleri. - Gilbert Strang. *Introduction to Linear Algebra*, 5th Edition. Wellesley-Cambridge Press, 2016. - Howard Anton, Chris Rorres. *Elementary Linear Algebra*, 12th Edition. Wiley, 2019. - David C. Lay, Steven R. Lay, Judi J. McDonald. *Linear Algebra and Its Applications*, 6th Edition. Pearson, 2021. - Gene H. Golub, Charles F. Van Loan. *Matrix Computations*, 4th Edition. Johns Hopkins University Press, 2013. - Lloyd N. Trefethen, David Bau III. *Numerical Linear Algebra*. SIAM, 1997. - Roger A. Horn, Charles R. Johnson. *Matrix Analysis*, 2nd Edition. Cambridge University Press, 2012. ## Cite This Work Köker, M. A. (2013). Linear Algebra: Matrices, Linear Systems and Eigenvalues. alikoker.com.tr. https://alikoker.com.tr/en/linear-algebra-matrices-linear-systems-eigenvalues - BibTeX: https://alikoker.com.tr/en/linear-algebra-matrices-linear-systems-eigenvalues.bib - RIS: https://alikoker.com.tr/en/linear-algebra-matrices-linear-systems-eigenvalues.ris - CSL-JSON: https://alikoker.com.tr/en/linear-algebra-matrices-linear-systems-eigenvalues.csl.json