Understand performance and scores
Read data readiness, offline performance, and campaign impact separately. Then use score semantics and signed reasons to interpret each result.
Read model performance
Training evaluates on later historical periods that were not used for fitting. Review the following before activation:
Target prevalence and class coverage
How common the event is, and whether both positive and negative subjects are sufficiently represented.
Ranking quality and lift
Whether high-ranked subjects concentrate more target events than the comparison population. Compare with a simple business-rule baseline.
Calibration
Whether estimated probabilities are consistent with observed event frequencies, overall and in important business groups.
Stability
Whether results remain useful across time, offers, channels, and new versus established subjects.
Compare results only for a consistent target, horizon, evaluation population, and time period. A rare event can have useful prioritization even when absolute probabilities are low. Readiness means a problem is trainable; it does not guarantee useful lift.
Offline lift is not incremental campaign lift. A controlled experiment is needed to establish whether an action changes the business outcome.
Read the score
score
Value from 0 to 1, increasing with modeled propensity. Interpret it as a percentage only with calibrated-probability semantics.
score_semantics
Distinguishes a calibrated probability from a ranking score.
reference_percentile
Position in the reference score distribution frozen at training time.
score_band
very_low, low, medium, high, or very_high, derived from the reference percentile rather than fixed probability intervals.
score_as_of
Business cutoff used to construct the score.
prediction_window_end
End of the future window covered by the score.
EPS publishes a calibrated probability only when calibration quality gates pass; it does not silently label an uncalibrated ranking as a probability.
For example, a 30-day score of 0.72 with score_semantics = calibrated_probability estimates a 72% probability of the defined event within that horizon. A reference_percentile of 0.92 places the subject above 92% of the training reference distribution, not necessarily today's eligible population.
Percentiles are not another probability. A reference-percentile threshold also does not guarantee a fixed number or percentage of current recipients. For a fixed campaign capacity, rank the currently eligible population and select the required count.
Read the reasons
Each score can have up to three signed local explanation families. The primary reason is the largest contribution by absolute value: it may raise or lower the score.
reason_family
Business grouping of the supporting signals.
reason_code
Stable activation code within that family.
direction
Whether the reason increases or decreases the target score.
reason_strength
Normalized strength for ordering reasons within the same score.
model_contribution
Signed local contribution to the model result.
population_deviation
Standardized difference from the training reference, when meaningful.
observed_value_summary, reference_value_summary
The subject's value and a comparable reference summary.
evidence_features
Underlying signals supporting the reason.
For a risk-focused policy, use direction = increases_target_score in the detailed reasons instead of assuming every primary reason is a risk driver. A high-risk subscription can have reduced usage increasing its score while its subscription channel lowers it.
Reasons describe local model contributions, not causes, universal correlations, p-values, or guaranteed treatment effects.
Reason families
Available families depend on the selected inputs. Subscription families cover behavior change, other behavior, lifecycle history, subscription terms, offer and price, and channel and distribution. Evidence can include recency, usage rhythm, prior cycles, tenure, commitment, discounts, and channel.
If no individual reason passes the explanation thresholds, the output can indicate a combination of weaker signals rather than inventing a dominant reason.
Warehouse outputs
A successful scoring publication provides a consistent batch across three logical outputs:
Current scores
Latest scores, percentiles, bands, top reasons, copied activation fields, and lineage.
Score history
Append-only scoring history for monitoring and analysis.
Score reasons
Up to three detailed signed reasons per score.
Subscription targets include subscription_id; profile-grain targets leave it empty. Model, run, and batch identifiers trace the output to its training. Check source freshness, score coverage, cutoff, and batch consistency before use.
Last updated