Linear Algebra for Machine Learning

🎯 Goal:

Help you understand and use vectors, matrices, and dot products β€” the building blocks behind ML models like linear regression, neural networks, and PCA.

🧱 1. What’s a Vector?

βœ… Definition:

A vector is just an ordered list of numbers β€” like a row or a column.

# A 3D vector (Python list)
v = [3, 5, 7]

In ML:

  • A feature vector for a house could be [bedrooms, size_in_sqft, price]
  • Images are flattened into long vectors of pixel values

🧰 2. What’s a Matrix?

βœ… Definition:

A matrix is just a grid of numbers (rows and columns).

# 2x3 matrix
A = [[1, 2, 3],
     [4, 5, 6]]

In ML:

  • A dataset with 3 features and 100 samples is a 100Γ—3 matrix
  • Neural network weights are stored as matrices

πŸ“ 3. Vector Operations

βœ… Scalar Multiplication:

2 * [3, 5, 7] = [6, 10, 14]

βœ… Vector Addition:

[1, 2] + [3, 4] = [4, 6]

These operations are used in gradient updates and feature scaling.

πŸ”„ 4. Matrix Multiplication (Very Important)

πŸ€” Why it matters:

This is how we pass data through a layer of a neural network.

# Matrix A (2x3) Γ— Vector x (3x1) = Result y (2x1)

A = [[1, 2, 3],
     [4, 5, 6]]
x = [1, 0, 1]

# y = A Β· x = [1Γ—1 + 2Γ—0 + 3Γ—1, 4Γ—1 + 5Γ—0 + 6Γ—1] = [4, 10]

🧠 Think of it as combining and weighting inputs.

πŸ’₯ 5. Dot Product (Core of Neural Networks)

βœ… Definition:

A dot product of two equal-length vectors is the sum of the product of corresponding elements.

a = [2, 3]
b = [4, 5]

# a Β· b = 2Γ—4 + 3Γ—5 = 8 + 15 = 23

In ML:

  • Used to calculate weighted sums in neurons
  • Forms the basis of similarity in NLP (cosine similarity)

πŸ“ˆ 6. Transpose

Changing rows into columns (or vice versa). Very common in reshaping data or aligning matrices.

A = [[1, 2],
     [3, 4]]

Aα΅€ = [[1, 3],
      [2, 4]]

🧠 7. Real ML Applications

ML ConceptLinear Algebra Behind It
Linear RegressionDot product of weights and inputs
Neural NetworksMatrix multiplications layer-by-layer
PCAEigenvectors of covariance matrix
Word EmbeddingsVectors capturing semantic similarity
Loss OptimisationGradient = vector of partial derivatives

πŸ“Œ TL;DR – What to Focus On First

βœ… Vectors & matrices
βœ… Dot product
βœ… Matrix multiplication
βœ… Scalar & element-wise operations

These four alone will get you 80% of the way in understanding how ML models compute.

Feedback Display

Learner Reviews