AI Engineering
AI Engineering, from Neurons to Agents
21How modern AI actually works: neural networks, transformers, fine-tuning, RAG, reinforcement learning, and agent engineering — explained from first principles with real math and code.
Foundations4 articles
Foundations
How LLMs Actually Work: The Next-Token Loop
How LLMs actually work — every model is a next-token loop of logits, temperature, softmax, top-k, top-p, and sampling, traced one token at a time.
8 min read
Foundations
From Hand-Written Rules to LLMs: How AI Learned to Learn
How AI moved from hand-written rules to learning from data — spam filters, word embeddings, attention, and the next-word loop behind ChatGPT, step by step.
9 min read
Foundations
Neural Networks from Scratch: How Machines Actually Learn
How neural networks learn from scratch — the neuron, non-linearity, activation functions, MSE loss, gradient descent, and backprop traced on a real XOR network.
10 min read
Foundations
Tensors and PyTorch: The Data Structure That Runs Deep Learning
How tensors, matrix multiplication, broadcasting, and PyTorch autograd power deep learning — with shapes, the dot product, and gradients traced by hand.
10 min read
Transformers & Architectures3 articles
Transformers & Architectures
Transformers, Part 1: Tokenization, Embeddings & Positional Encoding
How transformers turn text into meaningful numbers: tokenization, Byte-Pair Encoding, embeddings, and sinusoidal positional encoding, with worked examples.
9 min read
Transformers & Architectures
Transformers, Part 2: Self-Attention & Multi-Head Attention
How self-attention works — softmax(QKᵀ/√dₖ)V computed on a real sentence, why we scale by √dₖ, and how multi-head attention splits d_model across heads.
9 min read
Transformers & Architectures
What Changed Since 2017: RMSNorm, SwiGLU, RoPE, KV Cache & GQA
How a 2025 LLM differs from the 2017 transformer — RMSNorm, SwiGLU, RoPE, the KV cache and GQA, explained with the real formulas and a worked example.
10 min read
Training & Fine-tuning4 articles
Training & Fine-tuning
Train Your First Language Model: From Raw Text to Generation
Train your first language model from scratch — a character tokenizer, the six-step training loop, overfitting, and temperature/top-k text generation.
9 min read
Training & Fine-tuning
LLM Fine-Tuning, Part 1: When to Fine-Tune & How LoRA Works
When to fine-tune vs prompt or RAG, why full fine-tuning needs ~4x the model in VRAM, and how LoRA's ΔW = B·A adapters cut trainable params 64x.
10 min read
Training & Fine-tuning
Supervised Fine-Tuning: How a Base Model Becomes an Assistant
Supervised fine-tuning turns a base LLM into an assistant via one masked training step: tokenize, forward pass, softmax, cross-entropy, and a gradient update.
10 min read
Training & Fine-tuning
RLHF vs DPO: How Models Learn Human Preferences
How RLHF and DPO teach language models human preferences — the InstructGPT pipeline, the KL leash against reward hacking, and the DPO loss decoded step by step.
11 min read
RAG & Retrieval3 articles
RAG & Retrieval
What Is RAG? Retrieval-Augmented Generation from the Ground Up
A ground-up guide to retrieval-augmented generation — why LLMs need it, chunking, embeddings, cosine top-k search, contextual retrieval, and the full pipeline.
9 min read
RAG & Retrieval
Advanced RAG: Hybrid Search, Reranking & Query Transformation
How advanced RAG beats naive retrieval — hybrid BM25 + vector search fused by Reciprocal Rank Fusion, cross-encoder reranking, HyDE, and PageIndex.
10 min read
RAG & Retrieval
Recursive Language Models: Context as a Variable
A recursive language model keeps its input as a variable and pokes at it with code — no context rot, inputs 100x past the window. Here's how RLMs work.
9 min read
Agents & Reinforcement Learning6 articles
Agents & Reinforcement Learning
RLVR: Reinforcement Learning with Verifiable Rewards, Explained
How RLVR trains LLMs to reason — test-time compute, verifiable rewards vs. hackable reward models, and GRPO's group-relative advantage, from o1 to DeepSeek-R1.
8 min read
Agents & Reinforcement Learning
RL Environments for LLM Agents: Where Models Practice
How RL environments train LLM agents — the agent loop, the dataset-rollout-rubric trinity, reward hacking, and build-once-use-four-ways, with worked numbers.
10 min read
Agents & Reinforcement Learning
From APIs to Agents: Function Calling and the ReAct Loop
How a plain chat completion becomes an agent — function/tool calling, the ReAct think-act-observe loop, and why stateless context makes prompt_tokens balloon.
9 min read
Agents & Reinforcement Learning
Agent = Model + Harness: Context Engineering and Evals
An AI agent is a model plus a harness. Learn the primitives that patch model deficiencies, context engineering, and pass@k vs pass^k evals.
10 min read
Agents & Reinforcement Learning
AI Agent Memory: Beyond RAG to State That Persists
How AI agent memory differs from RAG — tracking state over time, the three ways a markdown memory file breaks, ADD/UPDATE/DELETE/NOOP writes, and decay.
8 min read
Agents & Reinforcement Learning
LangGraph Explained: Agents as State Machines
How LangGraph turns the agent while-loop into a graph — nodes, edges, shared state, per-key reducers, the tools loop, and checkpoint time travel.
9 min read
Practice & Craft1 article
Keep exploring
DSA & Algorithm Deep Dives
Keep exploring
System Design Guides