MLOps (Deep Workflow)
MLOps connects research velocity to production reliability: version data, code, and artifacts together; monitor behavior after deploy.
When to Offer This Workflow
Trigger conditions:
- First production model; batch or online serving
- Drift, bias, or latency SLO misses
- Compliance needs for lineage and explainability
Initial offer:
Use six stages: (1) problem & risk class, (2) data & reproducibility, (3) training & evaluation, (4) packaging & deployment, (5) monitoring & feedback, (6) governance & rollback). Confirm batch vs real-time and regulatory tier.
Stage 1: Problem & Risk Class
Goal: Align ML to decision risk (credit, health vs recommendation).
Exit condition: Offline and online success metrics defined.
Stage 2: Data & Reproducibility
Goal: Snapshot training data; deterministic pipelines; PII handling.
Practices
- Feature stores optional but valuable for consistency
- Secrets not in notebooks; orchestrated jobs
Exit condition: Run id reproduces artifact hash within agreed bounds.
Stage 3: Training & Evaluation
Goal: Train/val/test without leakage; time-series splits careful.
Practices
- Model card with limits and metrics
- Fairness slices where policy requires
Stage 4: Packaging & Deployment
Goal: Immutable artifacts; canary or shadow before full cutover.
Practices
- Model + preprocessing code version pinned together
Exit condition: Rollback to previous artifact id documented.
Stage 5: Monitoring & Feedback
Goal: Data drift, concept drift, latency; business KPIs tied to model decisions.
Practices
- Human review queue for low-confidence predictions when needed
Stage 6: Governance & Rollback
Goal: Approvals for retrain/deploy; audit trail; A/B for big changes.
Final Review Checklist
- Offline metrics aligned with business risk
- Data and code reproducibility
- Packaged artifacts with versioning and rollback
- Online monitoring and drift strategy
- Governance and approval path
Tips for Effective Guidance
- Training-serving skew is a top bug—feature parity tests help.
- Offline accuracy ≠ online business outcome.
- Fairness needs explicit slices—not one headline number.
Handling Deviations
- LLM-heavy products: lean on eval harnesses and prompt versioning (see llm-evaluation).
- Tiny teams: start with artifact registry + dashboards before a full feature store.