Skip to content

Success metrics (KPIs, guardrails, and exit criteria)

This page explains Success metrics (KPIs, guardrails, and exit criteria) and how it fits into the RecSys suite.

Who this is for

  • Stakeholders defining “what success means” for a RecSys pilot
  • Product + analytics teams setting measurement and guardrails

What you will get

  • A practical KPI + guardrail template you can reuse
  • A minimal set of exit criteria that prevents “shipping on broken data”
  • Links to the exact evaluation workflow pages

For each surface you pilot:

  1. Pick one primary KPI that represents business value.
  2. Pick 2–4 guardrails that must not regress.
  3. Define success criteria and rollback criteria up front.

Example KPI choices:

  • ecommerce: conversion rate, revenue per session, add-to-cart rate
  • content: completion rate, time spent, return rate

Common guardrails:

  • latency and error rate
  • empty-recs rate
  • join-rate / instrumentation integrity (for measurement)

Exit criteria (minimum to call the pilot “credible”)

  • You can validate logs against schemas (no “unknown fields” surprises).
  • Join integrity is sane (broken joins invalidate metrics).
  • You can produce at least one report that compares baseline vs candidate.
  • You have practiced rollback once (config/rules and/or manifest pointer).

How we measure (suite-level workflow)

Start here:

Then use the recsys-eval workflow pages: