How Modern AI Works: A No-Math Mental Model for PMs

Why PMs Need a Mental Model (Not a PhD)

As an AI PM you won’t train models yourself, but you will make decisions that depend on understanding how AI behaves: what it can do, where it breaks, what it costs, and why it sometimes confidently lies. This lesson gives you a durable, no-math mental model.

Traditional Software vs. Machine Learning

Traditional software is rules written by humans: if X then Y. A developer specifies every behavior. It’s predictable and auditable, but it can’t handle messy, fuzzy problems (e.g., “is this photo a cat?”).

Machine learning flips this. Instead of writing rules, you show examples and let the system infer the rules. You give it thousands of labeled cat/not-cat photos, and it learns the pattern. The output is a model—a learned function—not hand-written logic.

Key shift: with ML you curate data and define objectives; the model discovers the rules. This makes behavior probabilistic, not deterministic.

Neural Networks: Pattern Machines

A neural network is a stack of simple math units (“neurons”) connected in layers. Each connection has a weight. During training, the model makes a guess, measures how wrong it was (the error), and nudges the weights to reduce that error—repeated millions of times. That nudging process is learning.

Mental model: imagine tuning thousands of knobs until the machine’s outputs match the examples. “Deep learning” just means many layers of knobs, which lets the network capture very complex patterns.

• More data + bigger networks → can learn richer patterns
• Garbage data → garbage patterns (bias in, bias out)
• The model only knows what it has seen; it generalizes from training data

Generative AI and LLMs: Predicting the Next Token

Large Language Models (LLMs) are neural networks trained on enormous amounts of text. Their core trick is shockingly simple: predict the next chunk of text (a “token”) given everything so far. Do this repeatedly and you generate sentences, code, and answers.

Because they learned statistical patterns of language, they’re great at fluent, plausible output—but they have no built-in notion of truth. They produce what is likely, not what is verified. This is why hallucinations (confident, wrong answers) happen.

• Tokens: text is split into pieces; pricing and limits are measured in tokens
• Context window: how much text the model can “see” at once (its working memory)
• Prompt: the input that steers the prediction
• Temperature: a knob for randomness/creativity vs. consistency

Training vs. Fine-Tuning vs. Inference

• Training: the expensive, one-time-ish process of learning weights from massive data. Done by big labs; costs millions.
• Fine-tuning: taking a pretrained model and adjusting it on your narrower data/task. Cheaper, but still an investment.
• Inference: running the trained model to get an answer. This is what your users trigger every time—and what you pay for per request.

Product implication: most AI features ride on inference against an existing model (often via API). Latency and cost scale with tokens and model size. Bigger model = smarter but slower and pricier.

Where AI Fits (and Where It Doesn’t)

Good fits:

• Fuzzy, pattern-rich tasks (summarizing, classifying, drafting, extracting)
• Tasks where “good enough” + human review is acceptable
• High-volume work that’s tedious for humans

Poor fits:

• Tasks demanding guaranteed correctness with no oversight (e.g., exact legal/medical facts unaided)
• Problems with little or biased data
• Cases where explainability or strict determinism is legally required

Failure Modes a PM Must Plan For

• Hallucination: fluent but false. Mitigate with grounding (retrieval), citations, and human-in-the-loop.
• Bias: reflects training data; test across user segments.
• Drift / staleness: model knowledge has a cutoff; the world changes.
• Non-determinism: same input can yield different outputs—affects testing and trust.

The PM Takeaway

Think of an LLM as a brilliant, fast, well-read intern who is occasionally confidently wrong and never says “I don’t know.” Your product job is to set up the guardrails, context, and review loops that make that intern reliably useful.