Model Governance: Who Approves What, And When?
Your models are making decisions. Do you know who signs off on those decisions, or when? Untamed model deployments are a liability waiting to happen.
Blog · Topic
MLOps
Deploying, versioning and monitoring ML models: canaries, registries, rollbacks, drift and evaluation.
23 posts
Orchestrating Multi-Model Pipelines: DAGs Are Just The Start
Stop treating multi-model AI pipelines as simple DAGs. Production demands dynamic orchestration that handles real-world complexity, not just theoretical flows.
The Retraining Schedule That Doesn't Exist
Your model is degrading. You just don't know it yet. It's time to treat retraining as a core operational primitive, not a reactive fire drill.
The Model Card Nobody Read
Model cards get written for compliance and ignored in production. Here's what they should actually contain — and why your on-call team needs them.
The Hyperparameter Nobody Revisits
That magic number from your first training run is still in production. You just stopped noticing it.
The Cold Start Nobody Warned You About
Scale-to-zero sounds great until your first real user hits a frozen model and bounces. ML inference cold starts aren't a footnote — they're a product decision.
The A100 Surplus Hiding in Plain Sight
As of May 2026, RunPod charged $3.29/hr for an A100 SXM; Vast.ai listed one for $0.44. That 7.5× gap isn't a pricing glitch — it's a market signal.
The Deployment You Can't Explain to Compliance
Can you say which version you shipped, what data trained it, and why it makes the decisions it makes? If not, it's a governance problem, not a compliance one.
The Model That Passed Eval and Failed in Production
Offline metrics look great. Production behavior is a disaster. This gap isn't bad luck — it's a design failure you can prevent.
The A/B Test You Never Finished
Half your production models are running inside experiments that nobody has looked at in months. That's not science — that's clutter with a p-value.
Canary Deployments for ML Models
Software teams ship canaries without thinking twice. ML teams ship full replacements and call it 'confidence.' Here's why that's backwards — and how to fix it.
The Model That Works on Your Machine
It runs fine locally. It breaks in staging. It fails silently in production. ML environment parity is not a nice-to-have — it's the job.
The Staging Environment That Lies to You
Your ML staging environment feels like safety. It isn't. Here's what it's hiding.
The Shadow ML Dependency
Your model works. Your pipeline is green. But something is hardcoded to a version you never wrote down. That's the shadow dependency — and it will break you.
LLM Versioning: The Problem Nobody Solves Until It's Too Late
Your model changed under your app. Your prompt changed under your users. And nobody noticed until something broke. Fix this before it happens to you.
The Observability Stack for ML in Production
You monitor your servers. You don't monitor your models. Here's what that's costing you.
Feature Stores: Overhyped or Underused?
Everyone has an opinion on feature stores. Most of them are wrong. Here's when you actually need one.
Retrain vs Fine-Tune: Stop Guessing, Start Deciding
Two different tools for two different problems. Picking the wrong one wastes months.
The Cost of No Rollback Plan
Rolling back a model means rolling back code, data, features and serving config together. Here's what a real ML rollback plan looks like.
Monitor Model Drift Before Your Users Do
Your model isn't broken — it's just quietly wrong. Here's how to catch drift before it becomes a support ticket.
CI/CD for ML Is Not the Same as CI/CD for Software
Your software pipeline won't save your ML system. Here's what actually needs to be different — and why copying your DevOps playbook is a trap.
Evals Are Your Test Suite Now
Unit tests don't cover AI behavior. If you're shipping models without eval suites, you're shipping blind.
The Model Registry Is Not Optional
Why every production ML team needs model versioning, eval tracking, and promotion workflows.