Pitfalls
Where this goes wrong
Correlation traps, survivorship bias, manipulated data, vendor claims, and the questions where the honest answer is that this data will not do it.
The Correlation Traps in Rank Analysis
Five specific ways rank data produces confident wrong conclusions, all of them common enough to appear in published analysis regularly.
Analysis
Manipulated Rank and Fake Signals
Rank can be bought, temporarily. Recognising the signature protects your analysis, and the reasons not to do it yourself are practical rather than moral.
Analysis
What Analytics Vendors' Claims Actually Mean
The tools in this space make specific claims about accuracy, coverage and history. What each one means in practice and which questions get evasive answers.
Checklist
First-Party Data Beats Everything Here
Rank analysis exists because sellers cannot see each other's numbers. For your own products it is a poor substitute, used anyway.
Analysis
Survivorship Bias in Bestseller Analysis
Every study of what top products have in common is drawn from a sample that excludes the failures. The conclusions are unsupported and they are published constantly.
Analysis
When Rank Data Is the Wrong Tool
A list of questions where this data should be declined, with what to use instead. Refusing well is what makes the answers you do give credible.
Reference
A Checklist for a Defensible Analysis
Fifteen checks, run before an analysis leaves your hands, covering data integrity, method and presentation.
Checklist
Services That Sell Rank, and Why to Avoid Them
An industry sells rank improvement and reviews. What is actually sold, what enforcement looks like, and why it destroys measurement.
Analysis
8 notes