Structured data

Survival analysis

time to churn, machine failure, loan default, patient outcome

When an event will happen, for rows where it has not happened yet.

Probability & Bayes

Maximum likelihood

Foundations

Hazard and survival functions are probability; censoring is handled through the likelihood.

Frame the problem

Frame

The target is a duration with censoring: many rows have not had the event yet. A binary label at a fixed horizon throws that away.

Sourcing & signal

Exploring the data

Hunting for leakage

Data & labels

Censoring must be recorded honestly. Rows that left the dataset for other reasons are not events.

Encode & scale

Create features

Represent

Time-varying covariates need a long format: one row per entity per period.

Split the data

Temporal splits & backtesting

Split

By entity, and temporal if the hazard changes over calendar time.

Dumb baseline

Survival analysis

Gradient boosting

Model

Kaplan-Meier curves by segment are often the whole answer. Cox for interpretable hazards. Random survival forests or boosting with a survival loss for accuracy.

Train

Nothing unusual here.

Regression metrics

Calibration

Error analysis & significance

Evaluate offline

Concordance index for ranking, calibration of survival curves at chosen horizons. Slice by cohort.

Evaluate online

Nothing unusual here.

Serving & release

Monitor & retrain

Ship & monitor

Predictions are curves, not numbers. Decide what the consumer needs: median time, or probability by a date.

React Flow