AI Logbook — Artificial Intelligence Learning Journey

GitHub Issues Discussions

01/03

Apr 12 — Apr 18

Apr 27

The Consequence of Curves: Binary Cross-Entropy

When implementing Log Loss in software, engineers must account for the strict mathematical limits of logarithms.

#Confidence vs. Distance in Classification#Log Loss and Non-Convexity#The Log(0) Black Hole and Epsilon

Apr 26

The Geometry of Probability: The Sigmoid Function

The Vulnerability:
Passing a massive negative integer (e.g., $z = -1000$ ) into the sigmoid function requires the CPU to calculate $e^{1000}$ . This number is astronomically large and exceeds the 64-bit floating-point memory limits of standard Python arrays, resulting in a RuntimeWarning: overflow encountered in exp.

#The Artificial Neuron and Binary Classification#Exponential Bounding and Squashing Functions#Floating-Point Overflow and Clipping

Apr 25

Month 1 Retrospective: The Glass Box Engine

Constructing a production-ready machine learning engine entirely from scratch establishes a transparent mathematical pipeline, bypassing black-box abstractions.

#System Architecture of a Regressor#Linear Algebra & Calculus Synthesis#From Scratch Implementation Review

Apr 24

The Capstone Comparison: Custom Engine vs. Scikit-Learn

Production libraries operate as "Black Boxes." They prioritize computational efficiency and abstraction, obfuscating the underlying linear algebra and calculus driving the model's predictions.

#The Glass Box vs. The Black Box#Algorithmic Convergence and Coordinate Descent#Evaluating Production Equivalency

Apr 23

Upgrading the Engine: The Regularization Suite

Machine learning architectures enforce a strict mathematical separation in how state variables are handled during training:

#Hyperparameters vs. Parameters#The Complete Regularized Update#Class Integration and The Polynomial Insight

Apr 22

The Calculus of Regularization: Ridge (L2)

Why does squaring the weights fix multicollinearity? It comes down to the geometry of exponents.

#The Rubber Band Effect#Penalizing Confidence#Updating the Gradient Math

Apr 21

Evaluation Metrics: The R-Squared Proof

Mean Squared Error (MSE) is great for the Gradient Descent loop, but it is terrible for human evaluation. An MSE of 15,000 doesn't mean anything unless you know the scale of the dataset. $R^2$ normalizes the error into a scale-free ratio, acting much like a percentage score.

#Proving Your Representation#The Coefficient of Determination (R^2)#Implementing Evaluation Metrics

Apr 20

Feature Engineering: Bending the Matrix

"Linear regression is not weak. It’s actually very powerful if you give it the right representation."

#Linear in Weights, Non-Linear in Features#Expanding the Feature Space#Engineering Curves and Cycles

Apr 19

Synthetic Data Generation: Simulating Reality

When generating this dataset, we intentionally injected a fatal flaw that ruins naive linear regression: Multicollinearity.

#Multicollinearity and Non-Linearity#The Target Function & Irreducible Error#Building the Smart Building Dataset

01/03

Apr 12 — Apr 18

April 2026