NumPy Linear Algebra Basics

Ever written a loop to multiply two lists element-wise, then realized it crawls on real data? When your dataset outgrows a spreadsheet, linear algebra is the language, and NumPy is the translator. Without it, operations that should take microseconds take seconds, and your code becomes unreadable. This lesson demystifies linear algebra with NumPy—the mathematical backbone of data science—so you can stop fearing matrices and start using them like a pro.

The problem this lesson solves

Data science is full of hidden linear algebra. Every time you train a regression model, compute a similarity score, or transform high-dimensional data, you're doing matrix math. The problem? Python lists don't do math. Try adding two lists with + and you get concatenation, not element-wise addition. Loops work, but they're slow, verbose, and error-prone.

Here's a common pain point: you need to compute the dot product of two vectors for a machine learning algorithm. In pure Python, you'd loop over indices, multiply, and sum:

# Pure Python – slow and clunky
v1 = [1, 2, 3]
v2 = [4, 5, 6]
dot = 0
for i in range(len(v1)):
    dot += v1[i] * v2[i]
print(dot)  # 32

This is error-prone—what if the lists have different lengths? And it gets slower as data grows. NumPy solves this with vectorized operations that run at C speed.

Core concept / mental model

Think of linear algebra as math on blocks of numbers. A vector is a 1D array (a row or column of numbers). A matrix is a 2D grid. Scalars are single numbers. Linear algebra describes how to add, multiply, and transform these blocks—just like arithmetic for numbers, but with its own rules.

NumPy's ndarray is the container for these blocks. Crucially, it's vectorized: operations apply to all elements at once, avoiding Python-level loops. You can think of it as a magic table where every row and column understands math.

Here's a visual analogy: a list is like a set of drawers, each holding one number. An ndarray is like a spreadsheet—columns, rows, and formulas that work across all cells simultaneously.

The core operations you'll use:

• Element-wise operations: +, -, *, /—applied to each component.
• Dot product: np.dot() or @—multiplies vectors/matrices to give a single number or another matrix.
• Transpose: flips rows and columns.
• Matrix inverse: solves Ax = b systems.
• Norms: measure vector length—critical for distance calculations.

How it works step by step

Let's break down the essential linear algebra operations with NumPy.

1. Creating arrays

First, install NumPy (pip install numpy) and import it. Then create vectors and matrices from lists:

import numpy as np

vector = np.array([1, 2, 3])
matrix = np.array([[1, 2], [3, 4]])

print(vector.shape)  # (3,)
print(matrix.shape)  # (2, 2)

2. Element-wise operations

Unlike lists, ndarrays support arithmetic directly:

v = np.array([1, 2, 3])
print(v + 10)       # [11 12 13]
print(v * 2)        # [2 4 6]
print(v + v)        # [2 4 6]

3. Dot product and matrix multiplication

The dot product is fundamental—it measures how two vectors align. For matrices, @ performs matrix multiplication (not element-wise *).

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
print(np.dot(a, b))   # 32
print(a @ b)          # 32 – same result

A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
print(A @ B)
# [[19 22]
#  [43 50]]

Pro tip: Use @ for readability—'matrix multiplication' at a glance.

4. Transpose and reshape

Transposing flips rows and columns. Reshaping changes dimensions (but respects memory order).

m = np.array([[1, 2, 3], [4, 5, 6]])
print(m.T)
# [[1 4]
#  [2 5]
#  [3 6]]

v = np.array([1, 2, 3, 4, 5, 6])
print(v.reshape(2, 3))
# [[1 2 3]
#  [4 5 6]]

5. Solving linear systems

Linear algebra's classic problem: solve Ax = b. With NumPy, use np.linalg.solve (no inverse needed—faster and more stable).

A = np.array([[3, 1], [1, 2]])
b = np.array([9, 8])
x = np.linalg.solve(A, b)
print(x)  # [2. 3.]

Hands-on walkthrough

Let's apply these concepts to a real data science task: computing pairwise distances in a dataset. Here's a practical example using the Euclidean distance formula.

import numpy as np

# Sample data: each row is a point (x, y)
data = np.array([
    [1, 2],
    [3, 4],
    [5, 6],
])

# Compute distance from origin for each point
norms = np.linalg.norm(data, axis=1)
print(norms)  # [2.236 5. 7.81]

# Compute pairwise distance matrix
from scipy.spatial.distance import pdist, squareform  # SciPy adds power, but NumPy alone is enough!
# Without SciPy, use broadcasting:
points = data
diff = points[:, np.newaxis, :] - points[np.newaxis, :, :]
distances = np.sqrt(np.sum(diff**2, axis=2))
print(distances)
# [[0. 2.828 5.657]
#  [2.828 0. 2.828]
#  [5.657 2.828 0.]]

Expected output: each row's norm and the distance matrix. Note how broadcasting (adding new axes) lets us subtract all pairs at once—no loops!

Compare options / when to choose what

When to choose: Use np.linalg.solve instead of inv for solving systems—it's faster and more stable. Prefer @ over np.dot for readability.

Troubleshooting & edge cases

• Shape mismatch: ValueError: shapes (2,3) and (2,2) not aligned — inner dimensions must match for @ (e.g., (3,2) with (2,3)).
• Wrong vs @: Using on matrices does element-wise multiplication, not matrix multiplication. If you get unexpected numbers, check for * where you meant @.
• List confusion: np.array([1,2,3]) * 2 works, but [1,2,3] * 2 gives [1,2,3,1,2,3]. Always convert to ndarray first.
• Floating-point issues: Inverses and norms can have tiny errors (e.g., 1e-16). Use np.isclose() to compare floats.
• Vector vs matrix confusion: np.array([1,2,3]) is a 1D vector; np.array([[1,2,3]]) is a row matrix. .T on a 1D vector does nothing—reshape first.

What you learned & what's next

You now understand the core of linear algebra with NumPy:

• The problem: Python lists are slow and clunky for math.
• The mental model: vectors and matrices as ndarrays, vectorized operations.
• Step-by-step: creating arrays, element-wise ops, dot products, transposes, solving systems.
• Hands-on: distances and pairwise matrices with broadcasting.
• Comparison: @ vs *, solve vs inv.

Next, you'll explore eigenvalues and eigenvectors—the key to principal component analysis (PCA) and dimensionality reduction. With these skills, you're ready to tackle the math behind machine learning.

Remember: Any time you find yourself looping over numbers, ask—'Can NumPy do this in one line?' Usually, yes.