Introduction to Basis and Dimension
Basis and dimension are fundamental concepts in linear algebra that provide the mathematical framework for understanding vector spaces. A basis is a set of vectors that both spans a vector space and is linearly independent, while dimension measures the "size" or "degrees of freedom" of the space.
Why Basis and Dimension Matter:
- Provide coordinate systems for vector spaces
- Essential for solving systems of linear equations
- Foundation for linear transformations and matrices
- Critical in computer graphics, machine learning, and physics
- Enable dimension reduction techniques like PCA
- Fundamental for understanding solution spaces
Real-World Analogy: Think of a basis as a set of "building blocks" from which you can construct any vector in the space. The dimension tells you how many independent building blocks you need.
For example, in 3D space, we need exactly 3 independent directions (x, y, z axes) to describe any point. These axes form a basis, and the dimension is 3.
Vector Spaces
A vector space is a collection of objects (vectors) that can be added together and multiplied by scalars (numbers), satisfying certain axioms.
For a set V to be a vector space over a field F (usually ℝ or ℂ), it must satisfy:
- Closure under addition: u + v ∈ V for all u, v ∈ V
- Closure under scalar multiplication: αv ∈ V for all α ∈ F, v ∈ V
- Commutativity: u + v = v + u
- Associativity: (u + v) + w = u + (v + w)
- Zero vector: ∃0 ∈ V such that v + 0 = v for all v ∈ V
- Additive inverse: For each v ∈ V, ∃(-v) ∈ V such that v + (-v) = 0
- Distributive laws: α(u + v) = αu + αv and (α + β)v = αv + βv
Common Vector Spaces:
- ℝⁿ: n-dimensional real coordinate space
- ℂⁿ: n-dimensional complex coordinate space
- Pₙ: Polynomials of degree ≤ n
- Mₘₓₙ: m × n matrices
- C[a,b]: Continuous functions on interval [a,b]
Vector Space Explorer
ℝ² Example: All ordered pairs (x, y) where x, y ∈ ℝ
Standard Basis: {(1,0), (0,1)}
Dimension: 2
Any vector (x, y) can be written as x(1,0) + y(0,1)
Linear Independence
A set of vectors {v₁, v₂, ..., vₖ} is linearly independent if the only solution to the equation:
is the trivial solution c₁ = c₂ = ... = cₖ = 0. Otherwise, the vectors are linearly dependent.
Step 1: Form matrix A with vectors as columns
Step 2: Row reduce A to row echelon form
Step 3: Check if there are any free variables
Step 4: If no free variables → linearly independent
If free variables exist → linearly dependent
Example 1: Are v₁ = (1,2,3), v₂ = (4,5,6), v₃ = (7,8,9) linearly independent?
Form matrix:
⎡1 4 7⎤
⎢2 5 8⎥
⎣3 6 9⎦
Row reduction shows the third row becomes all zeros → linearly dependent
Example 2: Standard basis vectors in ℝ³: e₁ = (1,0,0), e₂ = (0,1,0), e₃ = (0,0,1)
c₁(1,0,0) + c₂(0,1,0) + c₃(0,0,1) = (0,0,0) ⇒ c₁ = c₂ = c₃ = 0
Thus, they are linearly independent.
Linear Independence Checker
Spanning Sets
The span of a set of vectors S = {v₁, v₂, ..., vₖ} is the set of all possible linear combinations of these vectors:
If Span(S) = V, then S is a spanning set for the vector space V.
- Span(S) is always a subspace of V
- If S ⊆ T, then Span(S) ⊆ Span(T)
- Adding a vector that's already in Span(S) doesn't change the span
- Removing a vector that's a linear combination of others doesn't change the span
Example 1: In ℝ², S = {(1,0), (0,1)} spans ℝ² because any vector (x,y) = x(1,0) + y(0,1)
Example 2: In ℝ³, S = {(1,0,0), (0,1,0)} spans the xy-plane (a 2D subspace of ℝ³)
Example 3: In P₂ (quadratic polynomials), {1, x, x²} spans P₂ because any quadratic a + bx + cx² is a linear combination
Step 1: Set up equation: v = c₁v₁ + c₂v₂ + ... + cₖvₖ
Step 2: Convert to system of linear equations
Step 3: Solve for coefficients c₁, c₂, ..., cₖ
Step 4: If solution exists → v ∈ Span(S)
If no solution → v ∉ Span(S)
Span Calculator
What is a Basis?
A basis for a vector space V is a set of vectors B = {v₁, v₂, ..., vₙ} that satisfies two conditions:
- Linear Independence: The vectors in B are linearly independent
- Spanning: The vectors in B span V (Span(B) = V)
A basis provides a "coordinate system" for the vector space. Every vector in V can be written uniquely as a linear combination of basis vectors.
Standard Basis Examples:
- ℝ²: {(1,0), (0,1)} is the standard basis
- ℝ³: {(1,0,0), (0,1,0), (0,0,1)} is the standard basis
- P₂ (quadratics): {1, x, x²} is the standard basis
- M₂ₓ₂ (2×2 matrices): { [1 0; 0 0], [0 1; 0 0], [0 0; 1 0], [0 0; 0 1] } is the standard basis
Minimal Spanning Set: A basis is the smallest set that spans the space
Remove any vector → no longer spans
Maximal Independent Set: A basis is the largest linearly independent set
Add any vector → becomes dependent
Unique Representation: Every vector has unique coordinates relative to a basis
v = a₁v₁ + ... + aₙvₙ with unique aᵢ
Not Unique: A vector space has infinitely many bases
But all bases have the same number of vectors
Alternative Basis for ℝ²: While {(1,0), (0,1)} is standard, {(1,1), (1,-1)} is also a basis.
Check: They're independent (not multiples of each other) and span ℝ² (any (x,y) = a(1,1) + b(1,-1) has solution a = (x+y)/2, b = (x-y)/2)
Finding a Basis
There are several methods to find a basis for a vector space or subspace. Here are the most common techniques:
Step 1: Write vectors as rows of a matrix
Step 2: Row reduce to row echelon form
Step 3: The non-zero rows form a basis for the row space
Step 1: Row reduce the matrix
Step 2: Identify pivot columns
Step 3: The original columns corresponding to pivot columns form a basis
Step 1: Solve Ax = 0
Step 2: Express solutions in parametric form
Step 3: The vectors multiplying free variables form a basis
Example: Find a basis for the column space of A =
⎡1 2 3⎤
⎢4 5 6⎥
⎣7 8 9⎦
Solution:
1. Row reduce:
⎡1 2 3⎤ → ⎡1 2 3⎤
⎢4 5 6⎥ → ⎢0 -3 -6⎥
⎣7 8 9⎦ → ⎣0 0 0⎦
2. Pivot columns: 1 and 2
3. Basis: {(1,4,7), (2,5,8)}
Basis Finder
Dimension of a Vector Space
The dimension of a vector space V, denoted dim(V), is the number of vectors in any basis for V.
All bases for a finite-dimensional vector space have the same number of vectors.
Common Dimensions:
- dim(ℝⁿ) = n
- dim(ℂⁿ) = n (over ℂ) or 2n (over ℝ)
- dim(Pₙ) = n + 1 (polynomials of degree ≤ n)
- dim(Mₘₓₙ) = m × n
- dim({0}) = 0 (zero vector space)
- If W is a subspace of V, then dim(W) ≤ dim(V)
- If dim(W) = dim(V) and W ⊆ V, then W = V
- Any set with more than dim(V) vectors is linearly dependent
- Any set with fewer than dim(V) vectors cannot span V
- Any linearly independent set with dim(V) vectors is a basis
- Any spanning set with dim(V) vectors is a basis
Method 1: Find a basis and count vectors
Method 2: For column space: rank of matrix
Method 3: For null space: number of free variables
Method 4: For solution space of Ax = 0: n - rank(A)
Example: Find dimension of subspace spanned by {(1,2,3), (4,5,6), (7,8,9)}
Solution: These vectors are linearly dependent (third is sum of first two). A basis is {(1,2,3), (4,5,6)}, so dimension = 2.
Rank-Nullity Theorem
The Rank-Nullity Theorem (also called the Dimension Theorem for linear transformations) is one of the most important results in linear algebra.
For any linear transformation T: V → W or any m × n matrix A:
where:
- rank(A) = dim(column space of A) = dim(row space of A)
- nullity(A) = dim(null space of A) = number of free variables
- n = number of columns of A = dimension of domain
Example: For A =
⎡1 2 3⎤
⎢4 5 6⎥
⎣7 8 9⎦
1. Row reduction shows rank(A) = 2
2. Number of columns n = 3
3. By Rank-Nullity: nullity(A) = n - rank(A) = 3 - 2 = 1
4. The null space is 1-dimensional (a line through origin)
Solving Systems: Determines if Ax = b has solutions
If rank(A) = rank([A|b]), system is consistent
Uniqueness: Determines if solution is unique
If nullity(A) = 0, solution is unique (if it exists)
Dimension Counting: Relates input and output spaces
rank(A) = dimension of output that's actually reached
Invertibility: Tests if matrix is invertible
A is invertible ⇔ rank(A) = n ⇔ nullity(A) = 0
Rank-Nullity Calculator
Applications of Basis and Dimension
Basis and dimension concepts are fundamental to many areas of mathematics, science, and engineering:
Computer Graphics
Basis vectors define coordinate systems for 3D modeling and rendering.
Example: Changing basis for camera views, object transformations, and texture mapping.
Dimension reduction techniques compress 3D models while preserving essential features.
Machine Learning
Principal Component Analysis (PCA) finds optimal basis for data representation.
Example: Reducing high-dimensional data to lower dimensions while preserving variance.
Basis functions in kernel methods and neural networks.
Signal Processing
Fourier basis decomposes signals into frequency components.
Example: JPEG compression uses cosine basis functions.
Wavelet bases provide multi-resolution analysis for images and signals.
Quantum Mechanics
State vectors live in Hilbert spaces with orthonormal bases.
Example: Choosing basis for observable measurements.
Dimension of space relates to degrees of freedom in quantum systems.
Problem: Compress a set of 1000 points in ℝ¹⁰⁰ (100-dimensional space) while preserving 95% of the variance.
Step 1: Compute covariance matrix of the data
Step 2: Find eigenvectors (principal components) - these form a new basis
Step 3: Keep only the k eigenvectors with largest eigenvalues
where k is chosen so that (λ₁+...+λₖ)/(λ₁+...+λ₁₀₀) ≥ 0.95
Step 4: Project data onto this k-dimensional subspace
Now data is represented in ℝᵏ where k < 100
Result: Data compressed from 100 dimensions to k dimensions while preserving most information.
Dimension Reduction: Original dim = 100, New dim = k, Compression ratio = (100-k)/100
Interactive Practice
Basis and Dimension Practice Tool
Practice basis and dimension concepts with randomly generated problems.
Select a topic and click "Generate Problem"
Solution:
1. Form matrix with vectors as rows and row reduce:
⎡1 2 3 4⎤ → ⎡1 2 3 4⎤ → ⎡1 2 3 4⎤
⎢2 4 6 8⎥ → ⎢0 0 0 0⎥ → ⎢0 -1 -2 -3⎥
⎢1 1 1 1⎥ → ⎢0 -1 -2 -3⎥ → ⎢0 0 0 0⎥
⎣3 5 7 9⎦ → ⎣0 -1 -2 -3⎦ → ⎣0 0 0 0⎦
2. Non-zero rows: (1,2,3,4) and (0,-1,-2,-3)
3. Basis: {(1,2,3,4), (0,-1,-2,-3)} or simpler: {(1,2,3,4), (1,1,1,1)}
4. Dimension = 2
Solution:
1. Row reduce A:
⎡1 2 1⎤ → ⎡1 2 1⎤ → ⎡1 2 1⎤
⎢2 4 2⎥ → ⎢0 0 0⎥ → ⎢0 0 1⎥
⎣3 6 4⎦ → ⎣0 0 1⎦ → ⎣0 0 0⎦
2. Pivot columns: 1 and 3 → rank(A) = 2
3. Number of columns n = 3
4. By Rank-Nullity: nullity(A) = n - rank(A) = 3 - 2 = 1
5. Verify: Solve Ax = 0 → x₁ + 2x₂ + x₃ = 0, x₃ = 0
Solution: x₁ = -2x₂, x₃ = 0 → Null space spanned by (-2,1,0)
Dimension of null space = 1 ✓
Basis and Dimension Summary & Reference
| Concept | Definition | Key Formula | Important Properties |
|---|---|---|---|
| Linear Independence | c₁v₁ + ... + cₙvₙ = 0 ⇒ all cᵢ = 0 | det(A) ≠ 0 for square matrices | Maximal independent set = basis |
| Span | All linear combinations of vectors | Span(S) = {Σcᵢvᵢ | cᵢ ∈ F} | Minimal spanning set = basis |
| Basis | Linearly independent spanning set | B = {v₁, ..., vₙ}, |B| = dim(V) | Not unique, all bases have same size |
| Dimension | Number of vectors in any basis | dim(V) = n | Invariant under isomorphism |
| Rank | dim(column space) = dim(row space) | rank(A) = # pivot columns | rank(A) ≤ min(m,n) |
| Nullity | dim(null space) | nullity(A) = # free variables | nullity(A) = n - rank(A) |
Mistake: Confusing row space and column space
Wrong: Basis from row reduction gives basis for row space, not necessarily column space
Correct: For column space basis, use original columns corresponding to pivot columns
Mistake: Assuming more vectors means larger dimension
Wrong: 100 vectors in ℝ² must span ℝ²
Correct: Maximum dimension in ℝ² is 2, regardless of how many vectors you have
Mistake: Forgetting the zero vector space
Wrong: Every vector space has a non-empty basis
Correct: The zero space {0} has dimension 0 and empty basis
Mistake: Misapplying Rank-Nullity
Wrong: rank(A) + nullity(A) = m (number of rows)
Correct: rank(A) + nullity(A) = n (number of columns)
- Visualize in low dimensions: Understand ℝ² and ℝ³ cases thoroughly before generalizing
- Check both conditions: For basis, verify both linear independence AND spanning
- Use row reduction systematically: It's the most reliable computational tool
- Understand the geometric meaning: Basis = coordinate axes, Dimension = degrees of freedom
- Practice with different fields: Work with ℝ, ℂ, and finite fields to deepen understanding
- Connect concepts: See how basis, dimension, rank, and nullity all relate through Rank-Nullity Theorem