Holdout Test — iGaming GlossaryA holdout test withholds a CRM campaign or bonus from a random subset of eligible iGaming players to measure the true incremental lift vs the treated group.

Holdout Test

**TL;DR:** A holdout test withholds a CRM campaign or bonus from a random subset of eligible iGaming players to measure the true incremental lift vs the treated group.

What it means

Most CRM reporting overstates impact because it credits campaigns with revenue that would have happened anyway. A holdout fixes that. Eligible players are randomly split — usually 80/20 or 90/10 — into treatment (receives the campaign) and control (does not). The difference in target KPI (FTD, NGR, retention) is the incremental impact.

Holdouts can be one-off (single campaign) or persistent (a permanent global control group of, say, 5% who never get any CRM). Persistent holdouts power CFO-grade reporting of CRM-attributed NGR.

Formula / How it's measured

Incremental Lift = (KPI in treatment) − (KPI in control), expressed per player.

Example: 50,000 dormant players in a Mexican casino are randomised 90/10. Treatment (45k) gets a 25 free spin reactivation; control (5k) gets nothing. 30-day NGR per player: treatment $9.20, control $4.10. Incremental = $5.10/player → 45,000 × $5.10 = $229,500 incremental NGR. Campaign cost: $80,000 → ROI 2.87×. Without the holdout, the operator would have credited the full $414,000 to the campaign.

Why it matters for operators

Holdouts are the only way to know which CRM activity actually creates value. Operators who institutionalise holdouts typically kill 20–40% of recurring campaigns and reinvest in winners, lifting overall CRM ROI by 30%+.

Common benchmarks (2026)

Persistent global holdout: 3%–10% of active players
Per-campaign holdout: 10%–20% of target audience
Minimum sample for stat significance: ~2,000 per arm for FTD KPIs
Test duration: 14–60 days depending on KPI maturity
Mature operators run holdouts on 70%+ of CRM campaigns