The Dot Product: Applying Importance with Weights

Weights and Linear CombinationsThe Dot Product FormulaImplementing the Dot Product in the Vector Class

🧠The Theory

AI/ML Concept: Applying Importance (Weights)

In machine learning, the dot product is how a model applies "importance" to different features. We call this level of importance a "weight".

When predicting a house price, not all features are equally important. Let's look at how a model might process this using a dot product:

Feature	Value (Vector x)	Weight (Vector w)	Product (x * w)
Bedrooms	3	50000	150000
Age (years)	15	-2000	-30000

The dot product combines them: $(3 \times 50000) + (15 \times -2000) = 120000$ . The model has used the dot product to calculate a mathematical prediction by scaling our data by its learned weights.

📐The Math

Math: The Dot Product

The dot product takes two vectors of the exact same dimension and multiplies them together to produce a single number (a scalar). We do this by multiplying the corresponding elements and adding up the results:

$\vec{a} \cdot \vec{b} = \sum_{i=1}^{n} a_i b_i = (a_1 \times b_1) + (a_2 \times b_2) + \dots + (a_n \times b_n)$

Unlike addition or subtraction which return a new vector, the dot product collapses the vectors down into a single value. Mathematically, this operation fundamentally measures how much two vectors "align" with each other in space.

⚙️The Code

import math

class Vector:
    def __init__(self, attributes: list[float]):
        self.attributes = attributes
    
    def __sub__(self, other: "Vector") -> "Vector":
        if isinstance(other, Vector):
            if len(self.attributes) != len(other.attributes):
                raise ValueError("Vectors must have the same dimension for subtraction.")
            return Vector([s - o for s, o in zip(self.attributes, other.attributes)])
        else:
            raise TypeError(f"Unsupported operand type for -: 'Vector' and '{type(other).__name__}'")


    def distance(self, other: "Vector") -> float:
        # 1. Calculate the difference vector
        diff = self - other

        # 2. Square each element in the difference vector
        squared_diff = [x ** 2 for x in diff.attributes]

        # 3. Sum the squared differences
        sum_squared_diff = sum(squared_diff)

        # 4. Take the square root of the sum
        distance = math.sqrt(sum_squared_diff)

        return distance

    def dot(self, other: "Vector") -> float:
        if isinstance(other, Vector):
            if len(self.attributes) != len(other.attributes):
                raise ValueError("Vectors must have the same dimension for dot product.")
            
            return sum([a * b for a, b in zip(self.attributes, other.attributes)])
        else:
            raise TypeError(f"Unsupported operand type for dot product: 'Vector' and '{type(other).__name__}'")

    def __repr__(self) -> str:
        attribute_str = ", ".join(f"{a:.2f}" for a in self.attributes)
        return f"Vector({attribute_str})"


# Example Usage: Predicting a house price using features and weights

# Features: [3 bedrooms, 2 bathrooms, 15 years old]
house_features = Vector([3, 2, 15])

# Weights: [50000 per bed, 25000 per bath, -2000 per year of age]
model_weights = Vector([50000, 25000, -2000])

# Calculate prediction using the dot product
base_prediction = house_features.dot(model_weights)
print(f"Base Price Prediction: ${base_prediction:.2f}")

Code Breakdown

def dot(self, other): We introduce the method to calculate the dot product between two vectors.
if len(self.attributes) != len(other.attributes): Just like in subtraction from What is a Vector? Translating the Real World into Code, the dot product is only mathematically defined if both vectors exist in the exact same dimensional space.
zip(self.attributes, other.attributes): We pair up the corresponding elements from our feature vector and our weight vector.
a * b for a, b in ...: We multiply each feature by its corresponding weight.
sum(...): We add all those individual products together to return a single scalar value, completing the $\sum_{i=1}^{n} a_i b_i$ formula.

Functions and Lines: Adding the Bias Euclidean Distance: Measuring Similarity