Databricks Certified Machine Learning Professional Study Guide

Issuing body: Databricks Jurisdiction: Global (vendor certification — not a legal license to practice) Level: Professional Prerequisites: None formal; 1+ year hands-on ML on Databricks recommended; the ML Associate is the natural feeder. (Source: catalogue record data-ai-ml-cert-012.) Renewal: 2 years (retake the current exam).

1. What this exam is

The Databricks ML Professional exam tests production ML on the Databricks Lakehouse: experiment tracking, model lifecycle, feature engineering, deployment, and MLOps — with MLflow as the backbone and Unity Catalog as the governance layer. Where the cloud-vendor ML exams (MLA-C02, GCP ML Engineer) test their platform's managed services, this one tests the lakehouse-native MLOps loop: data in Delta, experiments in MLflow, models in Unity Catalog, serving via Model Serving or batch. It's the MLOps-engineer credential for the Databricks ecosystem.

2. Pathway and exam format

Typical path: Databricks ML Associate (the fundamentals: notebooks, AutoML, basic MLflow) → 1+ year building ML on Databricks → this exam. The Data Engineer Professional pairs well (the data platform underneath).
Format: ~59 scored questions (per the Sept 2025 blueprint cited in the catalogue), 120 minutes, English only, $200 USD. Domains: Model Development, MLOps, Model Deployment (per catalogue record data-ai-ml-cert-012, verified against 2026 guides). Proctored online.
The exam's mental model: the ML lifecycle on the lakehouse — experiment (tracking), register (model registry), deploy (serving/batch), monitor (drift, quality) — with Unity Catalog governing every artifact.

3. Knowledge domains (with theory)

Public blueprint (approximate): Experimentation (~30%), Model Lifecycle/MLOps (~35%), Deployment (~20%), Model Monitoring/Serving concepts (~15%). Approximate; verify against the official exam guide.

Experimentation: MLflow tracking, done right

MLflow Tracking (the theory): experiments → runs; each run logs params (hyperparameters), metrics (evaluated over time — logged per epoch/step for curves), tags, and artifacts (models, plots, data snapshots). The discipline the exam tests: reproducibility = code version (git) + data version (Delta time travel / dataset hash) + params + environment (conda.yaml / container). A run without the data version is not reproducible.

Model signatures and flavors: the signature (input/output schema — enforced at serving, catching train/serve skew mechanically) and flavors (python_function, sklearn, pytorch, transformers...) — the serving layer loads the right flavor. Log the signature always — unsigned models are a production incident waiting to happen (schema drift passes silently).

Evaluation practice: train/validation/test discipline (test touched once), cross-validation for small data, time-based splits for temporal data (random splits leak the future — the exam's favorite methodology trap), metric selection by business cost (same precision/recall/F1 theory as the other ML guides).

Databricks extras: AutoML (baseline models + notebooks — the starting point, not the finish), hyperparameter tuning (Hyperopt/SparkTrials — Bayesian optimization distributed over the cluster).

Feature engineering on the lakehouse

Feature Store (Databricks): versioned features in Delta tables with point-in-time lookups (as-of joins for training — no future leakage) and online stores for low-latency serving. The exam's core scenario: training features computed in batch but needed at serving in real time — the feature store is the consistency mechanism (same definitions, two latencies).

Feature engineering practice: encoding (one-hot/target/with care — target encoding leaks without cross-validation), scaling (for regularized/NN models; trees don't care), time-based features (lags, rolling windows — with strict cutoff discipline), text embeddings. Data leakage taxonomy (memorize): target leakage (feature derived from the label), temporal leakage (future in training), preprocessing leakage (imputation/scaling fit on all data before splitting — fit only on train).

Model lifecycle: registry, lineage, governance

MLflow Model Registry → Unity Catalog: models registered with versions, stages/aliases (Champion/Challenger or Staging/Production — aliases are the modern pattern), and descriptions. Unity Catalog integration is the exam's governance story: models as securable objects (GRANT/REVOKE), lineage (which tables trained which model version — automatic), audit logs. The professional workflow: experiment → register candidate → evaluate (Champion vs Challenger) → alias promotion via CI/CD → deploy.

CI/CD for ML: Databricks Asset Bundles / dbx / GitHub Actions: on merge, run tests (data validation, model evaluation gates), register, and promote. Model validation gates (accuracy thresholds, fairness slices, latency budgets) block bad promotions mechanically.

Deployment: serving patterns

Model Serving (Databricks): serverless real-time endpoints (autoscaling, provisioned throughput option), A/B testing via traffic splitting, serverless GPU for GenAI. Batch inference (Spark UDF / pandas UDF over Delta tables — the high-throughput offline answer). The decision: latency requirement × volume × cost — same theory as every serving decision, with Databricks' two primitives (Serving vs batch).

GenAI on Databricks: Mosaic AI / Agent Framework concepts (RAG with Vector Search, agent evaluation via MLflow), model serving for LLMs (provisioned throughput for foundation models), AI Gateway (rate limiting, fallbacks, guardrails). The exam's GenAI questions test RAG-on-lakehouse (Vector Search over Delta) and evaluation — not model architecture.

Monitoring and maintenance

Lakehouse Monitoring: inference tables (payload logging — every prediction logged with inputs), drift detection (feature/label distribution vs baseline), quality monitoring (labels vs predictions as ground truth arrives), bias/fairness slices. Retraining triggers: drift alarms, schedule, performance decay — orchestrated via Workflows.

The feedback-loop hazard: predictions influencing future training data (e.g., fraud model suppresses the fraud it would have learned from) — monitor for label censoring, not just drift.

4. Key formulas and numbers

Same ML metric set as the other guides (precision/recall/F1/AUC-PR/RMSE) — plus calibration (predicted probabilities matching empirical rates — critical when scores drive decisions; check reliability diagrams).
Point-in-time correctness: feature as-of event_timestamp, never max(feature_timestamp) — the join condition is feature_ts <= event_ts per entity, latest before the event.
Serving cost: endpoint-hours × price + requests; batch = cluster-hours only. Break-even math per workload.
Experiment scale: Hyperopt trials ≈ 10–50× single-run cost; budget accordingly (tuning is the hidden ML cost).
Data versioning: Delta time travel (VERSION AS OF / TIMESTAMP AS OF) — free reproducibility for the training snapshot.

5. Worked examples

Example 1 — Leakage postmortem. Scenario: Model shows AUC 0.95 offline, 0.62 in production. Features include "days_since_last_login" computed from the full user table. Solution: Temporal leakage — "days_since_last_login" at training time was computed with data after the prediction point (the table kept updating). Fix: point-in-time feature computation (Feature Store as-of joins: features as of the event timestamp), time-based splits for evaluation, and a leakage audit (feature importance + temporal stability checks). Lesson: any feature touching mutable state is guilty until proven innocent.

Example 2 — Champion/Challenger deployment. Scenario: New gradient-boosting model claims +3% revenue; must prove it before full rollout. Solution: Register challenger in Unity Catalog; MLflow evaluation (offline: revenue-weighted metrics on recent data); Model Serving traffic split 90/10 with inference-table logging; monitor business metrics (revenue per session — not just AUC) for 2 weeks; promote challenger → Champion alias via CI/CD if the lift is significant (statistical test on the business metric, not eyeballing). Rollback = alias flip (seconds).

Example 3 — Feature store for fraud. Scenario: 200 features, batch-computed hourly; serving needs <50 ms decisions. Solution: Feature Store: offline (Delta, hourly batch for training with as-of correctness) + online store (DynamoDB-backed or Databricks online tables — <10 ms lookups); streaming feature pipeline (Structured Streaming) for the real-time features (velocity counters); serving joins online features at request time. The exam's pattern: define once, serve twice (batch for training, online for inference).

Example 4 — GenAI RAG on the lakehouse. Scenario: Support copilot over 100k docs in Delta; need eval + governance. Solution: Vector Search index over the docs Delta table (embeddings pipeline in a Workflow); agent built with the agent framework; MLflow evaluation (groundedness, relevance on a golden set); Unity Catalog governs the docs table, the index, and the model (lineage: which docs → which answers); AI Gateway for rate limits + guardrails. The lakehouse advantage: the same governance (UC) covers data, index, and model.

Example 5 — Retraining pipeline. Scenario: Demand-forecast model decays quarterly (seasonality); manual retraining is 2 weeks of toil. Solution: Databricks Workflow: monthly trigger → data validation (expectations on the new data) → training (Hyperopt, logged to MLflow) → evaluation gates (WAPE vs champion on holdout; fail → alert, don't promote) → register + alias promotion → batch inference refresh. Human approval gate before production alias change (the exam's governance answer). Toil eliminated; the 2 weeks become 2 hours of review.

6. Standards and regulations cross-reference

EU AI Act — model documentation (MLflow model cards/descriptions), eval artifacts, human oversight, logging (inference tables) — the lakehouse already produces most of the evidence; the exam-relevant point is which artifacts satisfy which obligation.
ISO/IEC 42001 / NIST AI RMF — governance wrappers (same as other ML guides).
Colorado SB 26-189 / California SB 53 — bias audits (sliced metrics in MLflow evaluation), documentation duties.
GDPR — training-data lineage (Unity Catalog: which tables → which model), erasure propagation (retrain or document the policy), DPIAs.
SR 11-7 / OCC 2011-12 — independent validation (the validation team gets MLflow/UC read access — separation of duties in the tooling).
FDA PCCP — for ML medical devices: the retraining pipeline is the change-control plan (pre-specified modifications, validation gates).

7. Pitfalls, traps, and exam strategy

Trap patterns:

The leakage trap: any offline/online performance gap → leakage audit before architecture changes. Time-based splits, as-of joins.
The signature trap: models logged without signatures — serving-time schema errors. Always log the signature.
The stage-vs-alias trap: the modern pattern is aliases (Champion/Challenger), not rigid Staging/Production stages. Know both; prefer aliases.
The batch-vs-serving trap: bulk offline scoring → batch inference (Spark), not a real-time endpoint. Same theory, Databricks primitives.
The UC trap: governance questions → Unity Catalog (lineage, grants, audit) — the exam's answer for "who accessed what, which data trained which model."
The AutoML trap: AutoML is the baseline, not the production model. The exam tests what comes after (tuning, validation, lifecycle).

Strategy: think lifecycle (experiment → register → deploy → monitor) with MLflow/UC as the spine. Memorize: the leakage taxonomy, point-in-time join semantics, Champion/Challenger mechanics, the serving decision (Serving vs batch), and Unity Catalog's governance objects. The GenAI questions reward RAG+eval systems thinking over model trivia.

8. Practice problems

P1. A churn model's features are computed nightly; at serving, the API computes them on the fly with slightly different code. Production AUC dropped 8 points. Diagnose and fix. Answer: Training/serving skew from dual implementations. Fix: Feature Store — single feature definitions, batch (offline) for training with as-of correctness, online store for serving lookups. Immediate: diff the two implementations (the bug is usually a default value or timezone). Durable: one codebase, two materializations.

P2. You must prove to auditors which data trained model version 7, who approved it, and who can access it. Name the artifacts. Answer: Unity Catalog lineage (tables → model v7, automatic), MLflow run (params, metrics, data version, code commit), model description/approval record (who promoted the alias, when), UC grants (who has EXECUTE/READ on the model), audit logs (access history). The exam's point: UC + MLflow produce the audit trail as a side effect of the workflow — if you used them.

P3. Design the eval gate blocking bad models from production. Specify the checks. Answer: On every candidate: (1) performance gate (primary metric ≥ champion − epsilon on holdout; epsilon small), (2) fairness slices (no group degrades >threshold vs champion), (3) latency gate (p99 inference < budget), (4) robustness (performance on recent drifted data), (5) schema/signature validation. All automated in CI/CD; failure → no alias promotion + alert. Human approval for the final production alias (governance).

P4. 10M rows/day need scoring; latency tolerance 4 hours; cost-minimal. Real-time endpoint vs batch? Answer: Batch inference (Spark): no endpoint-hours, scales to the backlog, ~10× cheaper per prediction at this volume. Real-time endpoints only if latency demanded it. Schedule via Workflows; write predictions to Delta (inference table for monitoring).

P5. A recommendation model's CTR decays over 6 weeks after each retrain, then jumps back after retraining. Explain and improve. Answer: Concept drift (user preferences shift; the model goes stale) + possible feedback loops (model narrows exposure, training data narrows further). Improve: (1) more frequent retraining (weekly, automated), (2) exploration (epsilon-greedy / bandit allocation for fresh items), (3) monitor drift continuously (Lakehouse Monitoring) instead of waiting for CTR decay, (4) incremental/online updates if the algorithm supports it. The decay-then-jump sawtooth is the signature of too-infrequent retraining.

P6. Your team fine-tunes an LLM on support tickets; legal asks about PII in training data and the EU AI Act. Give the engineering response. Answer: (1) PII: Sensitive-Data-Protection-style scrubbing of the ticket corpus before training (or train on de-identified data); document the scrubbing (audit artifact). (2) EU AI Act: support agent = limited-risk (transparency: disclose AI); if it makes consequential decisions, assess high-risk duties. (3) Artifacts: dataset lineage (UC), eval results (groundedness, safety red-team), model card (intended use, limitations), inference logging. (4) Ongoing: monitor for PII regurgitation (extraction evals). The exam's lesson: compliance is a pipeline property, not a document.

Original ETS reference writing, verified against public sources on 2026-09-30. Thin sourcing is flagged inline. Not legal advice — confirm current editions with the issuing body.

Databricks Certified Machine Learning Professional Study Guide