Matrix Addition & Scalar Multiplication

Manipulating the DatasetScaling and Shifting MatricesImplementing __add__ and __mul__

🧠The Theory

AI/ML Concept: Manipulating the Dataset

Why do we need these operations? In machine learning, we rarely use raw data exactly as it comes to us. We need to preprocess it so our neural networks can digest it effectively.

Scalar Multiplication (Scaling): Imagine our dataset has a column for "Square Footage" where values are in the thousands (e.g., $2500$ ) and another for "Bedrooms" (e.g., $3$ ). Large numbers can overwhelm an AI model and make training unstable. We use scalar multiplication to scale entire datasets down (e.g., multiplying the matrix by $0.001$ ) so the AI can process the data smoothly.
Matrix Addition (Shifting/Bias): When processing entire batches of data at once, we still need to add our Bias ( $b$ ) to our predictions, just like we did in Week 1. Matrix addition allows us to add that base bias across thousands of predictions simultaneously.

📐The Math

Math: Scaling and Shifting Matrices

1. Scalar Multiplication:
A "scalar" is just a fancy mathematical term for a single, standard number (like $5$ , or $-2.5$ ). When you multiply a matrix by a scalar, you simply multiply every single element inside the matrix by that number.

If $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$ , then $2A = \begin{bmatrix} 2 & 4 \\ 6 & 8 \end{bmatrix}$ .

2. Matrix Addition:
To add two matrices together, you simply add the elements that are in the exact same positions.

Crucial Rule: Because you match elements by position, you can only add matrices together if they have the exact same shape!

If $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$ and $B = \begin{bmatrix} 10 & 10 \\ 10 & 10 \end{bmatrix}$ , then $A + B = \begin{bmatrix} 11 & 12 \\ 13 & 14 \end{bmatrix}$ .

⚙️The Code

class Matrix:
    def __init__(self, data: list[list[float]]):
        if data:
            self.__validate(data)
            self.data = data
            self.number_of_rows = len(data)
            self.number_of_cols = len(data[0])            
        else:
            self.data = []
            self.number_of_rows = 0
            self.number_of_cols = 0

    def __validate(self, data: list[list[float]]) -> None:
        """Private method to ensure matrix is a perfect rectangle."""
        number_of_cols = len(data[0])
        for row in data:
            if len(row) != number_of_cols:
                raise ValueError("All rows must have the same number of columns to form a valid matrix.")

    @property
    def shape(self) -> tuple[int, int]:
        """Returns the shape of the matrix as (rows, columns)."""
        return (self.number_of_rows, self.number_of_cols)
    
    def __mul__(self, scalar: float) -> "Matrix":
        """Scalar multiplication: scales every element by the scalar."""
        return Matrix([[element * scalar for element in row] for row in self.data])

    def __add__(self, other: "Matrix") -> "Matrix":
        """Matrix addition: adds elements of identically shaped matrices."""
        if isinstance(other, Matrix):
            if self.shape != other.shape:
                raise ValueError("Matrices must have the same shape for addition")
            return Matrix([
                [a + b for a, b in zip(row1, row2)]
                for row1, row2 in zip(self.data, other.data)
            ])
        else:
            raise TypeError(f"Unsupported operand type for +: 'Matrix' and '{type(other).__name__}'")

    def __repr__(self) -> str:
        """Helper to print the matrix cleanly in the terminal."""
        rows_str = "\n  ".join(str(row) for row in self.data)
        return f"Matrix(\n  {rows_str}\n)"


# --- Example Usage ---

dataset = Matrix([
    [2000.0, 3.0], # House 1: 2000 sqft, 3 beds
    [1500.0, 2.0], # House 2: 1500 sqft, 2 beds
])

# 1. Scalar Multiplication (Scaling down the data)
scaled_dataset = dataset * 0.1

print("Original Dataset:")
print(dataset)
print("\nScaled Dataset (multiplied by 0.1):")
print(scaled_dataset)

# 2. Matrix Addition (Shifting data)
shift_matrix = Matrix([
    [10.0, 10.0], 
    [10.0, 10.0]
])

shifted_dataset = dataset + shift_matrix

print("\nShifted Dataset (+ 10 to all elements):")
print(shifted_dataset)

Code Breakdown

def __validate(self, data): Extracted validation logic into a private method to keep the initialization clean.
def __mul__(self, scalar): Overloads the * operator. It uses a nested list comprehension to iterate through every row, and every element in that row, multiplying it by the scalar.
def __add__(self, other): Overloads the + operator.
if self.shape != other.shape: The mathematical safeguard guaranteeing we only add matrices of identical dimensions.
zip(self.data, other.data): A highly optimized Python technique to pair up corresponding rows, and then corresponding elements within those rows, to perform the addition.

Matrix-Vector Multiplication: The Batch Forward Pass What is a Matrix? Representing Datasets