Applied AI Engineering

Learning objectives

• Explain why ML work happens in notebooks, and what "reproducible" means concretely (pinned environment, seeded randomness, re-runnable top-to-bottom).
• Vectorize a computation with NumPy instead of looping, and say why the array version is 10–100× faster.
• Load, clean, join, group, and reshape tabular data in pandas — handling missing values, wrong dtypes, and duplicates deliberately.
• Decide when to reach for NumPy vs pandas vs a plain Python loop.
• Run an EDA: produce labelled Matplotlib/Seaborn plots that answer a stated question, not just decorate the page.
• Set up a clean virtual environment and a notebook that another engineer can clone and re-run to the same numbers.

Key concepts

• The notebook is a REPL you can save
• Environments and reproducibility
• NumPy: think in arrays, not loops
• Pandas: a spreadsheet with an API
• Plotting and EDA — look before you model

Assignment

Clean a messy real-world CSV (Titanic or NYC-Airbnb) in pandas and ship a notebook that answers 3 stated questions with 5 labelled plots.

Feeds

P1 Insight Report

Primary tools

• NumPy
• pandas
• Matplotlib
• Seaborn
• Jupyter

Prerequisites

programming fluency

Learning objectives

• Explain a vector and a matrix as data structures, and read a dot product as a similarity / alignment score.
• Explain why "a model is matrix math" — why one matmul does thousands of dot products at once, and why shapes have to line up.
• Explain a derivative as sensitivity and a gradient as the steepest-uphill compass, and state the gradient-descent update rule from memory.
• Read a loss surface as terrain and describe what the learning rate controls.
• Summarize a dataset with mean and variance, standardize a feature, and say why scaling matters.
• Apply Bayes' rule to update a belief with evidence, and explain why correlation is not causation.
• Implement gradient descent from scratch to fit a line and check it against the closed-form least-squares answer.

Key concepts

• Linear algebra: many numbers at once
• Calculus: which way is downhill
• Probability & statistics: reasoning under uncertainty
• Putting it together: the learning loop

Assignment

Implement gradient descent from scratch in NumPy to fit a line to noisy data; plot the loss curve converging and check your slope against the closed-form (normal-equation) solution.

Primary tools

• NumPy
• Matplotlib

Prerequisites

01

Learning objectives

• Classify a problem as supervised, unsupervised, or reinforcement learning, and name what plays the role of "features" and "labels."
• Explain the paradigm shift from deterministic code to learned-from- data behavior, and why that makes generalization — not correctness on the training data — the thing you actually care about.
• Build a leak-free three-way split (train / validation / test) and say precisely what each slice is allowed to touch.
• Diagnose overfitting vs underfitting from a train-vs-validation gap, and reason about the bias–variance trade-off behind it.
• Walk the end-to-end ML lifecycle — problem → data → model → eval → deploy → monitor — and locate any task on it.
• Spot data leakage in a workflow and describe the fix, before it inflates a score you'll later have to retract.

Key concepts

• From rules to examples: the paradigm shift
• The three flavors of learning
• Features and labels
• Why we hide data from our own model
• Overfitting, underfitting, and the gap
• The bias–variance trade-off
• The end-to-end ML lifecycle

Assignment

On one dataset, make a leak-free 3-way split and produce a validation-curve plot that visibly shows under- and over-fitting as model complexity increases.

Primary tools

• scikit-learn

Prerequisites

01, 02

Learning objectives

• Distinguish regression, classification, clustering, and dimensionality reduction, and route a new problem to the right family.
• Explain each core algorithm in one line — linear/Ridge/Lasso, logistic regression, k-NN, Naive Bayes, decision trees, Random Forest, gradient boosting, k-Means, hierarchical clustering, PCA.
• Name each algorithm's key hyperparameters and its characteristic failure mode, so you can debug a bad result instead of guessing.
• Decide when to reach for a linear model, a tree ensemble, or an unsupervised method — and justify the choice.
• Argue why gradient boosting and Random Forests are the workhorses for tabular data, and name the cases where a humble linear model still wins.
• Assemble a five-model, cross-validated leaderboard and read it honestly (including when the top models are statistically tied).

Key concepts

• The lay of the land
• Regression: predicting a number
• Classification: predicting a category
• Ensembles: the tabular workhorses
• Unsupervised learning: no labels required
• The decision guide — when to use what

Assignment

Train five models on one problem, produce a cross-validated leaderboard table, and write two sentences on why the winner won.

Feeds

P2 Model Leaderboard

Primary tools

• scikit-learn
• XGBoost/LightGBM

Prerequisites

02, 03

Learning objectives

• Explain why accuracy is misleading on imbalanced data and name a metric that isn't.
• Build and read a confusion matrix, and derive precision, recall, and F1 from it by hand.
• Decide, for a given business problem, whether precision or recall matters more, and tune the decision threshold to hit a stated rule.
• Interpret ROC-AUC and PR-AUC, and know which one to trust when classes are imbalanced.
• Pick the right regression metric (RMSE, MAE, R²) and say what each penalizes.
• Always compare against a baseline (DummyClassifier / DummyRegressor) and run cross-validation correctly.
• Detect and prevent data leakage, and recognize the inflated-score smell before it ships.

Key concepts

• Accuracy is a liar on imbalanced data
• The confusion matrix: the source of truth
• Precision, recall, and F1 — derived from the matrix
• The threshold is a dial, not a constant
• Regression metrics: RMSE, MAE, R²
• Baselines are mandatory
• Cross-validation: one split lies to you
• Data leakage: the #1 way you fool yourself

Assignment

Tune a classifier's decision threshold to a business rule ("catch 90% of fraud"), report the precision/recall trade-off, then deliberately introduce a data leak and show the inflated-then-corrected score.

Feeds

P3 Cost-Sensitive Classifier

Primary tools

• scikit-learn

Prerequisites

03, 04

Learning objectives

• Impute missing values with the right strategy (mean, median, mode, or a missingness indicator) and explain why "just drop the rows" is usually wrong.
• Encode categoricals three ways — one-hot, ordinal, and target encoding — and say which one fits which situation, including target encoding's leakage trap.
• Scale features with standard, min-max, or robust scaling, and decide which models actually need it (and which couldn't care less).
• Engineer new features from existing columns and measure each one's lift on a validation metric rather than guessing.
• Handle an imbalanced target with class weights vs resampling (over/under-sampling, SMOTE), compare them on honest metrics, and know why resampling must happen inside the cross-validation fold.
• Assemble every fit-transform into a Pipeline so preprocessing is learned on the training data only — the single habit that prevents most leakage.

Key concepts

• The shape of the work: wrangle, then engineer
• Missing values
• Encoding categoricals
• Scaling & normalization
• Feature engineering: the highest-leverage work
• Imbalanced data
• The golden rule: fit on training data only

Assignment

A6

Feeds

P4 Feature Lab

Primary tools

• pandas
• scikit-learn

Prerequisites

01, 05

Learning objectives

• Explain a neuron as a weighted sum plus a nonlinearity, and see that logistic regression is a one-neuron network.
• Explain why a nonlinear activation is mandatory — that without it, any number of stacked layers collapses back into a single linear model.
• Compute a forward pass as a sequence of matrix multiplies and know the shape of every intermediate tensor.
• Choose a loss function (MSE for regression, cross-entropy for classification) and say why cross-entropy is the right call for classes.
• Explain backpropagation as the chain rule — "gradient descent through composed functions" — and write the gradient formulas for a 2-layer net.
• Build a 2-layer MLP in pure NumPy with manual backprop, train it to high accuracy on a toy set, and reproduce it in PyTorch.

Key concepts

• From a straight line to a neuron
• Why you need a nonlinearity
• The forward pass is just matrix multiplies
• Loss: turning wrongness into a number
• Learning = gradient descent
• Backprop: gradient descent through composed functions
• Putting it together: the training loop

Assignment

A7

Feeds

P5 NN From Scratch

Primary tools

• NumPy
• Keras/PyTorch

Prerequisites

02, 04

Learning objectives

• Explain what an LLM is in one sentence — a next-token predictor trained on huge text — and why that framing predicts its strengths and failures.
• Distinguish tokens from words, and estimate how many tokens a chunk of text will cost.
• Reason about the context window as finite working memory and predict when a task will overflow it.
• Control temperature and explain what it trades off — and know which models still expose that knob.
• Call the Claude API from Python: send a prompt, read the reply, read token usage, and estimate the dollar cost.
• Describe what an embedding is and why it is the bridge to semantic search and RAG in Level 2.
• Decide when a task suits an LLM at all versus a classical model.

Key concepts

• What an LLM actually is
• Tokens, not words
• The context window: finite working memory
• Temperature: the randomness knob
• Calling the API: a prompt is just the input
• What embeddings are (a taster for Level 2)

Assignment

Build a CLI that summarizes any text file via the Claude API; test 3 prompt variants × 2 temperatures and log how the outputs change (and what they cost).

Primary tools

• Anthropic Claude SDK

Prerequisites

01, 07

Learning objectives

• Explain why a notebook is not a shippable product, and what a UI and a URL each add.
• Install and run an open model locally with Ollama, and call it from Python through its OpenAI-compatible endpoint.
• Decide honestly when to run a local model (cost, privacy, offline) versus a hosted API (capability), and name the limits of each.
• Point one client at many providers by swapping base_url and model, and reason about routing and fallback.
• Build a Streamlit app: understand the top-to-bottom rerun model, use st.session_state, wire input widgets to model output, build a chat UI, and cache a loaded model with st.cache_resource.
• Choose between Streamlit and Gradio for a given demo, and build the same demo in Gradio's Interface.
• Deploy a Streamlit app to a public URL and name the real caveats (secrets, cold starts, cost).

Key concepts

• A notebook is not a product
• Running open models locally with Ollama
• One interface, many providers
• Building a UI with Streamlit
• Gradio: the ML-demo alternative
• A taste of deployment

Assignment

Wrap your Level 1 model in a shareable Streamlit app with an input form, a prediction/answer display, and one LLM-powered feature — a natural-language explanation of the prediction via the Claude API.

Primary tools

• Ollama
• **Streamlit**
• Gradio
• Anthropic SDK

Prerequisites

01, 08

Prerequisites

01–09

Key outcomes

• ship an end-to-end tabular ML app with an LLM feature, evaluated honestly

Assignment & project

Capstone review

Primary tools

• the L1 stack

Learning objectives

• Write the anatomy of a PyTorch training loop from memory — zero_grad → forward → loss → backward → step — and say what each line does and what breaks if you drop it.
• Explain SGD, momentum, and Adam, and decide which to reach for.
• Set a learning rate deliberately, and explain schedules (step, cosine) and warmup and when each earns its place.
• Explain backpropagation as reverse-mode autodiff over a computational graph, and why it costs memory proportional to network depth.
• Apply dropout, batch normalization, early stopping, and weight decay, and say what each regularizes and when not to use it.
• Move a model and its data to a GPU correctly, and explain why it's faster.
• Run the "training is broken" checklist — diagnose a flat loss, exploding/vanishing gradients, and overfitting, and apply the fix.

Key concepts

• The anatomy of a training loop
• From full-batch descent to SGD
• Optimizers: SGD, momentum, Adam
• The learning rate — your most important knob
• Backprop in depth: reverse-mode autodiff
• Regularization: making it generalize
• Using GPUs
• Debugging training: the checklist

Assignment

A (CNN ablation)

Primary tools

• PyTorch
• Keras

Prerequisites

L1-07

Learning objectives

• Explain convolution as a small learned filter slid across an image, and say why that beats a fully-connected layer for pixels.
• Compute the output shape of a conv layer from its kernel size, stride, and padding, and explain receptive field and feature maps.
• Describe the arc from LeNet → VGG → ResNet and explain, concretely, why residual (skip) connections let networks go very deep.
• Choose between feature extraction and fine-tuning for a transfer- learning task, and justify the choice from your data size and domain.
• Fine-tune a pretrained ResNet on a custom 3-class set, report accuracy, and surface and explain its mistakes.
• Explain at a high level how object detection works — bounding boxes, IoU, and the difference between YOLO and Faster R-CNN — and where vision transformers fit.

Key concepts

• Convolution: a learned sliding filter
• Stride, padding, and the receptive field
• Pooling and the shape of a convnet
• From LeNet to VGG to ResNet
• Residual connections: why ResNet works
• Transfer learning: the professional default
• Beyond classification: detection in one page

Assignment

Fine-tune a pretrained ResNet on a custom 3-class image set (~300 images), report accuracy, and show 5 misclassifications with a hypothesis for each.

Feeds

P6 Transfer-Learning Classifier

Primary tools

• PyTorch
• torchvision
• timm

Prerequisites

01

Learning objectives

• Explain the RNN / LSTM / GRU family and name their four structural limits: the sequential bottleneck, vanishing gradients, no parallelism, and weak long-range memory.
• Explain attention as a soft, content-based lookup, and read the roles of query, key, and value (Q/K/V).
• Write the scaled dot-product attention formula from memory and say what each piece does, including why you divide by √dₖ.
• Explain multi-head attention, positional encoding, and the difference between encoder-decoder and decoder-only stacks.
• Explain tokenization (BPE/subword), the token-vs-word distinction, embedding tables, and what a context window costs.
• Say precisely why the Transformer parallelizes and an RNN cannot — and why that one fact reshaped the field.

Key concepts

• The old way: recurrent networks and their four limits
• The core idea: attention
• Scaled dot-product attention, the formula
• Why divide by √dₖ
• Multi-head attention
• Order is missing: positional encoding
• Assembling the Transformer
• Tokenization and embeddings, in depth
• Why it parallelizes (and beats the RNN)

Assignment

Implement scaled-dot-product self-attention by hand in PyTorch on a toy sequence, visualize the attention matrix, and explain in 3 sentences why it beats an RNN here.

Feeds

P7 Attention, Explained

Primary tools

• PyTorch

Prerequisites

01

Learning objectives

• Explain the shift from bag-of-words to embeddings, and why embeddings capture meaning that keyword matching cannot.
• Decide when to bother with classical preprocessing (tokenization, normalization, stopword removal) and when the embedding era makes it unnecessary.
• Distinguish word embeddings (word2vec/GloVe) from sentence embeddings (sentence-transformers) and say why you almost always want the latter.
• Build semantic search: embed a corpus, embed a query, rank by cosine similarity, return the nearest documents.
• Use Hugging Face pipeline for text classification, zero-shot classification, and named-entity recognition (NER) without training anything.
• Compare semantic search against a TF-IDF keyword baseline and articulate, with evidence, where each one wins and loses.
• Connect all of this to RAG — recognize that a semantic search index is a retriever waiting for a generator.

Key concepts

• The big shift: from counting words to capturing meaning
• Text preprocessing — and when not to bother
• From word embeddings to sentence embeddings
• Cosine similarity and semantic search
• The Hugging Face task menu
• Semantic vs keyword: where each actually wins
• This is the retrieval half of RAG

Assignment

Build semantic search over 500 documents with sentence embeddings + cosine similarity, build a TF-IDF keyword baseline, run 5 queries, and present a side-by-side top-5 comparison with commentary on where semantic wins and where it loses.

Feeds

P8 Semantic Search Engine

Primary tools

• Hugging Face Transformers & Datasets

Prerequisites

03

Learning objectives

• Compose a request from its parts — model, system prompt, messages, max_tokens — and explain what each one controls.
• Apply the three core prompt patterns — zero-shot, few-shot, and chain-of-thought — and say when each earns its cost.
• Explain temperature and sampling honestly, including why the parameter is removed on the current Opus and Sonnet models and how to steer them instead.
• Get structured (JSON) output two ways — by prompting, and by the API's structured-output / tool-calling feature — and explain why only the second guarantees the shape.
• Wire up tool / function calling: define a tool, run the call loop, and feed the result back.
• Name the four main failure modes — hallucination, format drift, refusals, truncation — and write the validator or check that catches each.
• Measure prompt reliability: build a test harness and report a metric (e.g. valid-JSON rate) so you can compare designs with numbers, not opinions.

Key concepts

• The anatomy of a request
• Prompt patterns: zero-shot, few-shot, chain-of-thought
• Temperature and sampling — the honest version
• Structured output: prompt-only vs guaranteed
• Tool / function calling
• When models fail — and how to catch it
• Measure it: prompting is an experiment

Feeds

P9 Structured-Output Extractor

Primary tools

• Anthropic Claude SDK

Prerequisites

L1-08, 03

Learning objectives

• Explain the full RAG pipeline — chunk → embed → store → retrieve → ground — and what each stage is responsible for.
• Decide when to reach for RAG versus fine-tuning versus a bigger context window, and articulate why RAG wins for knowledge.
• Chunk a corpus sensibly and explain the trade-off between fixed-size, sentence-aware, and overlapping chunks.
• Retrieve the top-k relevant chunks with embeddings and cosine similarity, and read what the similarity scores are telling you.
• Ground an answer in retrieved context with inline citations, and force the model to abstain when the answer isn't there.
• Name RAG's three failure modes — bad chunking, retrieval miss, ignored context — and know which metric catches each.
• Evaluate a RAG system on both halves: retrieval quality (recall@k, MRR) and answer faithfulness.

Key concepts

• RAG in one picture
• Why RAG beats fine-tuning for knowledge
• Step 1 — Chunking: cut the corpus into retrievable pieces
• Step 2 — Embeddings and the vector store
• Step 3 — Retrieval: the top-k nearest chunks
• Step 4 — Grounding: answer from the retrieved context
• The three failure modes
• Evaluating a RAG system

Assignment

A (minimal RAG

Feeds

feeds capstone)

Primary tools

• Chroma/FAISS
• LangChain/LlamaIndex
• Claude SDK

Prerequisites

04, 05

Learning objectives

• Decide between prompting, RAG, and fine-tuning by classifying the gap as knowledge vs behavior/format vs style — and justify it out loud.
• Explain what fine-tuning does mechanically: it keeps training a pretrained model's weights on your examples, changing the weights themselves.
• Explain why full fine-tuning is expensive and how PEFT / LoRA makes it cheap — low-rank adapters that train ~1% of the parameters.
• Describe embedding fine-tuning and when improving the retriever beats touching the generator.
• Compare the three methods on cost, latency, quality, and data needs, and read a decision-memo table.
• Run a small LoRA fine-tune of a tiny classifier locally and read its real numbers — trainable-parameter count, train time, accuracy, adapter size.
• Name the failure mode of each method and when not to use it.

Key concepts

• The three levers
• The decision framework
• Fine-tuning basics
• PEFT and LoRA: why low-rank adapters make fine-tuning cheap
• Embedding fine-tuning: sharpen the retriever, not the generator
• Cost, latency, quality, data: the four axes that decide it
• A concrete three-way result

Feeds

P10 Prompt vs RAG vs Fine-tune

Primary tools

• HF PEFT/LoRA
• Claude SDK

Prerequisites

05, 06

Learning objectives

• Explain why GenAI evaluation is hard — no single ground truth, open-ended outputs, non-determinism — and what that changes about how you measure.
• Define hallucination, faithfulness/groundedness, answer relevance, and correctness, and say which one a given failure belongs to.
• Build a gold set (eval dataset) for a RAG app and distinguish offline from online evaluation.
• Design an LLM-as-judge evaluator with a scored rubric, a forced rationale-before-verdict order, and a structured verdict.
• Name the judge's own biases — position, verbosity, self-preference — and apply concrete mitigations.
• Validate the judge itself against human labels before you trust its numbers.
• Add basic guardrails: input/output filtering, PII redaction, refusal handling, and jailbreak awareness.

Key concepts

• Why GenAI evaluation is genuinely hard
• The vocabulary: what "good" decomposes into
• Gold sets: your eval dataset
• Offline vs online evaluation
• LLM-as-judge: the core technique
• The judge has biases too — name them and mitigate
• Validate the judge before you trust it
• Guardrails and safety basics

Feeds

P11 RAG Eval Harness

Primary tools

• Anthropic SDK
• Ragas/promptfoo

Prerequisites

06, 07

Learning objectives

• Explain the difference between a discriminative and a generative model, and give an example of each.
• Describe an autoencoder as a learned compress-then-reconstruct pair, and say what its latent space (bottleneck) represents.
• Explain why a plain autoencoder is a poor generator and how a variational autoencoder (VAE) fixes this by making the latent space smooth and samplable.
• Explain the GAN setup — a generator and a discriminator locked in a minimax game — and describe mode collapse and why GANs are unstable to train.
• Explain diffusion models as iterative denoising, and articulate why they overtook GANs for high-quality image generation.
• Contrast VAE vs GAN vs diffusion on training stability, sample quality, speed, and typical use case.
• Explain at a conceptual level how multimodal models like CLIP put images and text in one shared embedding space, and why that unlocks text-to-image generation.
• Train a small autoencoder, visualize reconstructions and a latent scatter, and read what the latent geometry is telling you.

Key concepts

• Discriminative vs generative: the core split
• Autoencoders: compress, then reconstruct
• VAEs: make the latent space samplable
• GANs: two networks in a duel
• Diffusion: generation by iterative denoising
• Multimodal: putting images and text in one space
• The three families, side by side

Assignment

A (autoencoder + latent space)

Primary tools

• PyTorch

Prerequisites

01

Learning objectives

• Explain what "multimodal" means and why the shared-embedding idea lets one representation space hold text, audio, and images.
• Transcribe audio with open-source Whisper (whisper / faster-whisper), including language detection and word timestamps, and name Whisper's failure modes.
• Generate speech from text with an open or hosted TTS model, and weigh the honest trade-offs between the options.
• Generate images with Hugging Face diffusers, steering output with the key knobs — steps, guidance scale, and a fixed seed — and reason about cost, latency, and safety.
• Send an image to Claude via the Anthropic SDK's image content blocks to describe, analyze, or extract structured fields from a picture or document, and name where vision models hallucinate.
• Build a multimodal app that chains modalities (audio → transcript → structured minutes, or image → question answering) behind a Streamlit or Gradio UI.

Key concepts

• What "multimodal" actually means
• Speech-to-text with Whisper
• Text-to-speech
• Generating images with diffusion
• Vision-language: sending an image to Claude
• Building a multimodal app

Assignment

Build a multimodal app end to end — upload audio → transcribe with Whisper → produce structured meeting minutes and action items with Claude, or upload an image → ask questions about it with Claude vision — with a small quality check on 3–5 examples and an honest note on where it fails.

Primary tools

• Whisper
• HF diffusers
• Claude vision
• Streamlit

Prerequisites

05, 09, L1-09

Prerequisites

11–20

Key outcomes

• ship a cited, tool-using RAG assistant with an eval harness in CI, deployed

Assignment & project

Capstone review

Primary tools

• the L2 stack

Learning objectives

• Explain what makes a system an agent — the model owns the control flow — and distinguish it from a single call or a hand-coded tool pipeline.
• Trace the agent loop (perceive → reason → act) and map it onto the Claude Messages API's stop_reason == "tool_use" cycle.
• Describe the ReAct pattern (interleaved reasoning and acting) and reflection, and say when each earns its cost.
• Distinguish short-term memory (the running context) from long-term memory (an external store), and connect long-term memory back to RAG.
• Build a single tool-using agent with a manual loop and an iteration cap, wiring a calculator tool and one web/API tool.
• Diagnose the three ways agents fail — looping, error compounding, and runaway cost/latency — and apply concrete mitigations.
• Decide when a task actually needs an agent versus a cheaper, more reliable single call or scripted workflow.

Key concepts

• What makes it an agent: the loop
• ReAct: reasoning and acting, interleaved
• Reflection and planning
• Memory: short-term vs long-term
• Why agents are less reliable than single calls
• A quick note on frameworks

Assignment

Build a single tool-using agent (ReAct loop) that answers questions needing a calculator + one API/web tool; log its reasoning trace, and show one task it solves cleanly and one it loops on — with an analysis of why it loops and how you'd fix it.

Feeds

P12 Tool-Using Agent

Primary tools

• Claude Agent SDK / MCP
• LangGraph

Prerequisites

L2-05

Learning objectives

• Explain the AI Capability Stack (L1–L8) and place any real system on it, including the insight that governance adds control, not capability.
• Distinguish the core orchestration patterns — supervisor/delegation, planner→executor, researcher→writer, debate/critique — and say when each fits.
• Build a multi-agent workflow as separate message threads, each with its own system prompt and (often) its own model tier.
• Diagnose why multi-agent systems fail: error compounding across steps, loops, and runaway cost and latency.
• Decide, honestly, when to add a second agent and when a single well-prompted agent (or plain code) is the better engineering choice.
• Estimate the extra cost and latency a second agent buys, and whether the quality lift is worth it.

Key concepts

• From one agent to many: the orchestration idea
• The AI Capability Stack (L1–L8)
• Orchestration patterns
• Why agents fail
• When multi-agent actually helps

Feeds

P13 Multi-Agent Workflow

Primary tools

• LangGraph
• CrewAI
• AutoGen

Prerequisites

01

Learning objectives

• Explain why the control layer lives in your code, outside the model's reasoning — and why that boundary is what makes a guardrail un-bypassable.
• Assess an agent action along the three risk axes — blast radius, reversibility, and least privilege — and pick a control that fits.
• Build a policy / guardrail that classifies each proposed action as allow, deny, or needs-review, defaulting to deny for anything unknown.
• Insert a human-in-the-loop approval gate that intercepts a dangerous tool_use before it executes, and route the human decision through a trusted channel the model can't touch.
• Design an append-only, tamper-evident audit trail and a PII-routing step, and say what each is for.
• Red-team your own guardrail with a prompt-injection bypass attempt and show, with output, that the action is blocked — the essence of agent validation.

Key concepts

• The one idea: the control layer lives outside the model
• Risk management: three axes for every action
• The control layer, mechanism by mechanism
• Agent evaluation & validation: prove the gate holds

Feeds

P14 Guardrailed Action

Primary tools

• Guardrails AI
• NeMo Guardrails
• Llama Guard

Prerequisites

01, 02

Learning objectives

• Account for the cost of any LLM call from its token usage, and explain why usage.input_tokens alone understates the real prompt size.
• Instrument an app with a tracer — DIY or Langfuse/LangSmith — that records tokens, latency, and cost per call, and aggregate those traces into cost/latency dashboards.
• Explain prompt caching as a prefix match, place a cache breakpoint correctly, and verify hits from the response's usage fields.
• Design an eval-in-CI gate that treats quality statistically (a pass rate over a gold set), so non-determinism doesn't let regressions through.
• Distinguish the flavors of drift (input drift, output/quality drift, prompt/version drift) and describe how to detect each in production.
• Decide when to reach for quantization, batching, a smaller model, or an A/B test — and what each trades away.
• Version prompts and models like code, so a quality change is always traceable to a specific change.

Key concepts

• The mental shift: from "is it up?" to "is it good, and what did it cost?"
• Token and cost accounting: the meter you must read
• Serving & inference optimization — where the wins are
• Prompt caching, done right
• Observability: tracing tokens, latency, and cost
• CI/CD for LLMs: evals in the pipeline
• Version management and prompt drift
• Monitoring & drift
• A/B testing

Feeds

P15 Observability Retrofit

Primary tools

• LangSmith
• Langfuse
• Phoenix
• MLflow

Prerequisites

L2-08

Learning objectives

• Explain why LLM inference lives on GPUs and why VRAM, not compute, is the binding constraint — and compute a model's memory footprint from its parameter count.
• Apply quantization (FP16 → INT8 → INT4) to fit a model on smaller hardware, and state what accuracy you trade for the space you save.
• Distinguish data parallelism from model (tensor/pipeline) parallelism at a conceptual level, and say which one a big model forces on you.
• Describe how a modern serving engine wins throughput: the KV cache, PagedAttention, and continuous batching — and pick between vLLM, TGI, and Ollama for a given job.
• Measure a serving setup honestly in throughput (tokens/sec), latency (p50/p95), and time to first token, and load-test it under concurrency.
• Decide self-host vs. hosted API with a break-even cost model, and know the utilization level below which the API always wins.

Key concepts

• Why LLMs live on GPUs
• VRAM is the constraint — and you can compute it
• Quantization: making the model fit
• Distributed training, briefly (and why serving differs)
• Model serving: the throughput problem
• Data & retrieval pipelines at scale
• Self-host vs. hosted API: the break-even

Assignment

Serve an open model with vLLM/Ollama, containerize it with Docker, load-test it under concurrency, and report throughput/latency vs a hosted API — with an honest cost/break-even analysis.

Feeds

P16 Self-Hosted Serving

Primary tools

• Docker
• vLLM/TGI
• Ray

Prerequisites

04

Learning objectives

• Explain why dense-only retrieval misses, and name the two failure classes it struggles with (vocabulary mismatch and exact-term matching).
• Build hybrid search: run BM25 and dense retrieval side by side and fuse their rankings with reciprocal rank fusion (RRF).
• Add a cross-encoder re-ranker as a second stage, and explain the bi-encoder-versus-cross-encoder trade-off that makes two-stage retrieval work.
• Decide when a question needs GraphRAG (multi-hop, relationship, and global questions) instead of flat vector search, and sketch the Neo4j version.
• Design an agentic RAG loop where retrieval is a tool the model calls iteratively, and say what it buys and what it costs.
• Choose between stuffing the context and retrieving, and apply context compression to send the model less but better.
• Measure the retrieval-quality lift of each upgrade with recall@k and MRR on a labelled gold set, and reject upgrades that don't pay for themselves.

Key concepts

• The ceiling of dense-only retrieval
• Hybrid search: dense meaning plus sparse keywords
• Re-ranking: read the query and chunk together
• GraphRAG: when the answer isn't in any single chunk
• Agentic RAG: retrieval in a loop
• Long-context strategies: stuff, retrieve, or compress

Assignment

A (hybrid + re-ranker, recall@k)

Primary tools

• rank_bm25
• cross-encoders
• Neo4j

Prerequisites

L2-06

Learning objectives

• Measure group fairness with disparate impact (the four-fifths rule) and explain why several fairness definitions can't all hold at once.
• Explain alignment at a high level — the helpful/harmless/honest goal, RLHF, and constitutional AI — and why an aligned model is still not a secure one.
• Distinguish the core attack classes: direct vs. indirect prompt injection, jailbreaks, and data exfiltration, and articulate the lethal trifecta that makes exfiltration possible.
• Design layered defenses — input/output filtering, privilege separation / least privilege, human-in-the-loop, and provenance / trust boundaries — and reason about what each one does and doesn't stop.
• Build a categorized adversarial test suite and a scoring harness that reports attack-success rate before and after mitigations.
• Place a system in the EU AI Act risk tiers and list the documentation and human-oversight obligations that attach to it.
• State honestly why no single defense is complete, and why defense-in-depth plus monitoring is the realistic posture.

Key concepts

• Two duties: responsible and secure
• Bias and fairness: measure before you moralize
• Alignment in one page
• The security mindset: your agent is an attack surface
• Defenses: assume the model will be fooled
• Compliance: EU AI Act and enterprise policy

Feeds

P17 Security Hardening

Primary tools

• Llama Guard
• guardrails tooling

Prerequisites

03

Learning objectives

• Explain scaling laws, compute-optimal training, and reasoning models in plain terms — and say what they do and don't let you predict.
• Hold a balanced, non-hype position on what "AGI" means and why timelines are contested, and describe a practical way to stay current without chasing press releases.
• Screen candidate AI use cases for high ROI and place them on a value/effort matrix.
• Run a structured build-vs-buy decision using weighted criteria and the AI Capability Stack.
• Build a simple ROI/TCO model — benefits, one-time and run costs, payback — and defend the numbers and their assumptions.
• Map AI opportunities across horizontals (HR, Finance, Support…) and verticals (Healthcare, Banking, Retail…) and pick the highest-value cell.
• Draft a 2-page AI opportunity brief that a decision-maker would fund.

Key concepts

• The frontier, honestly
• From capability to value
• Mapping AI to verticals and horizontals

Assignment

A (2-page opportunity brief)

Primary tools

• (analytical)

Prerequisites

all

Prerequisites

23–30

Key outcomes

• build, deploy, and defend an agentic product with a business case

Assignment & project

Defense (diploma gate)

Primary tools

• the full stack