When Rank Data Is the Wrong Tool
A list of questions where this data should be declined, with what to use instead. Refusing well is what makes the answers you do give credible.
An analytics function that answers every question with the data it has loses the ability to be believed. These are the cases where the honest answer is that rank data will not do it.
Investment decisions and valuations
Why not: the estimation error is a factor of two or more, and it compounds through revenue, growth and share calculations.
Use instead: audited financials, industry data, direct disclosure in diligence. Rank data can be a sanity check on claims and cannot support a valuation.
Legal claims
Why not: the method is unpublished, the conversion basis is volunteered data, and the error range is wide. It will not survive expert challenge.
Use instead: discovery, first-party records, marketplace-provided data obtained through proper process.
Quantities for ordering
Why not: the error range exceeds what an inventory decision can absorb.
Use instead: first-party sales history, with rank as a directional and seasonal input.
Your own product performance
Why not: you have the actual units.
Use instead: your own data, with rank alongside for market context and calibration.
Profitability of any product
Why not: no cost information is observable.
Use instead: nothing external. This is not obtainable.
Total market size in a long-tailed category
Why not: the tail dominates the count and is where estimates are worst; errors are correlated and do not average out.
Use instead: industry data, a bounded top-of-category estimate with the scope stated, or relative sizing.
Anything about a specific individual or company's intentions
Why not: rank shows movement, never cause.
Use instead: public statements, filings, hiring, product changes. Rank can corroborate a narrative and cannot generate one.
Products in the deep long tail
Why not: rank there is a record of the last sale and its decay. The noise floor exceeds any effect you would want to measure.
Use instead: review velocity as a coarser but steadier proxy, or accept that the question is unanswerable externally.
Very short-term questions
Why not: the update cadence and the noise floor mean a few days of data supports nothing.
Use instead: wait, or use first-party data if it is your product.
Cross-marketplace performance comparison
Why not: ranks from different catalogues are not comparable.
Use instead: relative position against a local basket in each, stated as a directional comparison only.
How to decline
Say what the data can support instead. "I cannot give you a market size; I can tell you the top fifty products' combined volume within a factor of two, and whether the category is growing."
Say what would answer it. Industry data, first-party records, a longer observation window, a different measure.
Say what it would cost to get closer, if anything would.
Do not produce a number with a disclaimer for a question in this list. The disclaimer will be dropped and the number will be used.
Why this matters more than the techniques
Every method on this site produces estimates with real uncertainty. A function known to decline the questions it cannot answer is believed when it answers the ones it can.
A function that always produces a figure is, correctly, discounted on all of them.
Offering the alternative
Declining well requires having something to offer, and the alternatives are usually available.
For quantity questions: first-party data if it is yours; industry data if it exists; a bounded estimate with explicit scope if neither.
For causal questions: an experiment you could run, described concretely with its cost.
For competitor internals: public filings, job postings, product announcements, patent activity, and what a customer conversation would reveal.
For market size: trade bodies, regulatory data, listed company disclosures, and relative sizing from rank as a supporting input.
For anything urgent that needs more time: a provisional answer with its confidence, and a date when a better one is available.
A refusal accompanied by a route to the answer is a service. A refusal on its own is an obstacle, and it is why analysts who cannot offer alternatives end up producing the numbers they should have declined.