LLMOps vs MLOps

The instinct to treat LLMOps as an entirely new discipline is mostly marketing. Versioning what you deploy, reproducing what you shipped, monitoring in production, rolling back safely, keeping a staging environment that resembles production — all of this is the same job with a different payload.

Engineers with real MLOps experience are well positioned. The gap is narrower than job descriptions suggest, and framing existing experience correctly matters more than starting over.

The same applies to the cultural parts. Knowing how to run an incident, how to argue for reliability work against feature pressure, and how to build a deployment process people actually follow transfers completely, and is harder to acquire than any of the new technical material.

In classical ML you trained the model, so you controlled when it changed. With a hosted LLM, the thing at the centre of your system is a dependency you do not version and cannot pin indefinitely. A provider update can shift behaviour without any change on your side.

That inverts a core assumption. Reproducibility now means capturing everything around the call — prompt, context, parameters, retrieved documents, model identifier — because the one component you cannot snapshot is the model itself.

It also makes provider migration a standing concern rather than a one-off. Teams that abstracted the call boundary early move between providers in days; teams that scattered vendor-specific calls through their codebase spend a quarter on it.

A classifier gives you accuracy on a held-out set. A summariser gives you text, and whether that text is good is a judgement. This is the single biggest practical difference, and it is where most teams underinvest.

The answer is building evaluation as infrastructure: a versioned set of cases with expected properties, run automatically, with results tracked over time. Without it there is no way to tell whether a prompt change improved things or whether last week was a fluke.

Getting started is less daunting than it sounds. Thirty real cases with expected properties, run on every change, catches more regressions than an elaborate framework nobody maintains — and the habit matters more than the sophistication.

• A held-out set of real cases, versioned like code
• Automated checks for the properties you can state precisely
• Model-graded evaluation for the ones you cannot, with spot checks
• Regression runs before any prompt, context or model change
• Production sampling reviewed by humans on a schedule

Classical ML concentrated cost in training: expensive, occasional, easy to see. LLM systems concentrate it in serving, where it scales with usage and hides inside a per-request token count that nobody looks at until the invoice arrives.

This makes token accounting an operational metric alongside latency and error rate. A context change that improves quality by two per cent and triples token count is a decision, not a free win, and the team should be able to see both numbers before shipping it.

The failure pattern is consistent: cost is fine in staging because volume is low, then becomes the largest line item within a month of launch. Instrumenting cost per request before launch rather than after is the cheapest thing on this list.

Prompts, retrieval indexes, tool definitions and evaluation sets all behave like production artefacts: they change behaviour, they need review, and a bad one causes an incident. Treating them as configuration edited in a console is how teams end up unable to explain a regression.

The mature setup keeps them in version control, reviews changes like code, and ties each deployment to a specific combination. When something breaks, the question "what changed" has an answer.

Retrieval indexes deserve particular care because they drift silently. An index rebuilt with a different embedding model or a changed chunking rule produces different answers with no deployment event attached, which is close to undiagnosable without versioning.

Classical ML had data leakage and bias concerns, which remain. What is new is that the system now consumes untrusted text and, increasingly, acts on it — which introduces failure modes with no counterpart in a classifier deployment.

Prompt injection is the headline one: content retrieved from a document or a web page can address the model directly, and no amount of instruction reliably prevents it. The operational answer is not better prompting but narrower capability — the component reading untrusted content should have no tool that does anything irreversible.

Output handling matters too. Text produced by a model and rendered into a page, passed to a shell, or used in a database query is the same category of risk as any other untrusted input, and it is routinely treated as though it were trusted because it came from your own system.

• Untrusted retrieved content reaching a component with tools
• Sensitive data leaking into context and then into logs
• Model output rendered or executed without escaping
• Prompts containing credentials or internal detail

The transferable core is your platform and reliability experience. What to add is evaluation design, retrieval systems, token and latency budgeting, and the safety surface — prompt injection, data leakage through context, output handling.

A convincing portfolio piece is small but complete: a deployed LLM feature with a versioned evaluation suite, cost per request tracked, and a written account of one regression you caught before release. That last part is what separates candidates who have operated these systems from those who have only built demos.

In interviews, lead with the operational instincts rather than the model knowledge. Being able to say how you would roll back a prompt change, how you would detect a silent index rebuild, or what you would instrument before launch is worth more than describing an architecture.