Measuring Whether a Promotion Worked
The peak is the least informative part. A design that distinguishes pulled-forward demand from genuine incremental sales, using data you already have.
Almost every promotional post-mortem measures the peak, which reflects a burst, and concludes success. The question worth answering is whether anything durable happened.
The design
A baseline period. At least a week before the promotion, ideally two, with no other changes. Characterise the normal range.
The promotion period. Whatever it is.
A decay period. Several days after the promotion ends, excluded from measurement, during which the rank effect of the promotional sales fades.
A post period. At least a week after the decay, compared against the baseline.
A control basket throughout, so market movement is separable.
What each period tells you
Baseline: the level you are trying to beat.
Promotion peak: the size of the burst. Interesting for capacity planning and nearly useless for evaluating effect.
Post period versus baseline: the actual question. Did the product settle at a higher level than before?
If the post period equals the baseline, the promotion produced sales during the promotion and nothing after. That may still be worthwhile — a stock clearance, a cash flow need, a launch push — but it is not the durable gain that usually gets claimed.
If the post period is below baseline, the promotion pulled demand forward from the following weeks, which is a real and common outcome.
If the post period is above baseline, something durable happened: new customers, improved search position, review accumulation.
The confounders to remove
Basket movement. If the whole category rose during the post period, the gain is not yours.
Price. If the price did not return fully to baseline, you are measuring a price change, not a promotion.
Availability. A stock-out during any period invalidates it.
Seasonality. A promotion run just before a seasonal rise will look durable and will not be.
Other activity. Advertising, listing changes, publicity. One change at a time, or the attribution is guesswork.
The metrics worth reporting
Baseline median rank and range.
Peak rank and its date.
Post-period median rank and range.
Basket movement over the same periods.
Estimated incremental units, as a range, computed from the post-period difference rather than from the peak.
Duration of elevated performance, which is frequently more informative than magnitude.
The pull-forward question
The hardest and most important part.
A promotion that sells three weeks of demand in three days has not created demand; it has moved it and discounted it.
The signature is a post period below baseline, recovering over the following weeks.
Detecting it needs a longer post period than most measurement allows — three to four weeks rather than one.
It is worth measuring once properly even if you cannot do it every time, because the answer for your category and your customers tends to be stable, and knowing it changes how promotions are planned.
What rank data cannot tell you here
Margin. A promotion that increased units and destroyed contribution is a success in rank data and a failure in the accounts.
Customer mix. Whether the incremental buyers were new or existing.
Cannibalisation of your other products.
These need first-party data, and a promotion evaluated on rank alone is evaluated on half the question.
The honest report
"Baseline median rank 8,200 (range 6,900–9,800). Peak 1,150 on day two. Post-period median 7,400 (range 6,500–8,600), measured 10 to 17 days after the promotion ended. The control basket moved 4 percent over the same period. Estimated incremental effect: modest and positive, roughly 10 percent improvement in settled velocity, with a wide range."
That is a paragraph a decision-maker can use. "Rank improved 86 percent during the promotion" is the same event described in a way that will produce another promotion with the same result and the same misreading.
A promotion measurement template
The fields to fill in for every promotion, so that a year's promotions become comparable.
Promotion type and mechanic. Price cut, coupon, deal placement, bundle.
Depth, as a percentage of the base price.
Duration.
Baseline period dates, with the median rank and range.
Peak rank and date.
Decay period dates, excluded from measurement.
Post period dates, with the median rank and range.
Basket movement across all three periods.
Estimated incremental units, as a range.
Whether the post period was above, at, or below baseline.
Filled in consistently, this accumulates into the most valuable dataset a marketing function can have: which promotional mechanics produce durable effects in your category and which pull demand forward. Nobody can answer that from a single promotion, and almost anyone can answer it from twenty.