Click-through prediction

The output feeds an auction, so it must be a calibrated probability, not just a ranking. Define the attribution window for conversions.

Position bias, and delayed feedback: conversions arrive days later. Decide how to label impressions whose window has not closed.

Very high-cardinality ids: hashing and embeddings, not one-hot. Crosses between user, ad, and context carry the signal.

Strictly temporal. Yesterday predicts today.

Logistic regression with feature crosses was the industry standard for a decade and is still hard to beat per unit of compute. Gradient boosting for tens of millions of rows. Factorization machines and deep rankers when crosses become unmanageable by hand.

Log loss, not AUC, is the objective. Recalibrate after any downsampling of negatives. Online updates for fast-moving inventory.

Log loss and calibration curves per slice. AUC for ranking quality. Normalised entropy against a predict-the-average baseline.

Revenue and calibration in the experiment. Off-policy evaluation from logged auctions.

Millisecond latency. Watch calibration drift daily; a miscalibrated model overbids.