App Store Charts and Other Ranking Systems
The same reasoning applies to app charts, streaming charts and book bestseller lists, with different mechanics that change what the number means.
Ranking systems appear wherever a catalogue needs ordering by popularity. The analytical discipline transfers; the mechanics differ enough to matter.
App store charts
What they measure. Download velocity, with recency weighting, and in most implementations additional inputs — retention, engagement, ratings, and in some cases revenue for separate grossing charts.
Separate charts for free, paid and grossing, which measure different things and are frequently confused.
Category and overall charts, with the same narrow-category dynamics described elsewhere on this site.
Regional charts, with each store region ranked separately.
Key difference from product marketplaces: downloads are frequently free, so chart position reflects acquisition rather than revenue. The grossing chart is the revenue proxy and it is a different ordering entirely.
Update cadence is generally faster than product marketplaces, and the top charts move visibly through a day.
Streaming and music charts
Blended methodologies. Most modern music charts combine streams, sales and sometimes radio, with weightings that are published and change.
Published methodology is the notable difference — unlike marketplace ranks, chart methodologies here are typically documented, which makes analysis considerably more tractable.
Stream weighting varies by subscription type in several charts, so the same play counts differently depending on the listener.
Gaming is a known problem and chart operators actively adjust methodology in response.
Book bestseller lists
Editorial and methodological discretion. Several prominent lists apply rules about what qualifies, exclude bulk purchases, and reserve editorial judgement.
They are not a pure sales ordering and treating them as one produces confusion when a high-selling title does not appear.
Reporting panels rather than complete data in some cases — a sample of retailers rather than the whole market.
Different lists disagree because they measure different populations. This is a feature of the methods, not an error in any of them.
What transfers from the marketplace material
Relativity. Position depends on everything else.
Recency weighting and decay.
The need for a control set.
Narrow category inflation. True everywhere there is a category hierarchy.
Seasonality.
Estimation requires calibration and calibrations do not transfer between systems.
What to check before analysing a new system
Is the methodology published? If so, read it. This is the single largest difference between systems and it changes the whole approach.
What is the input — units, revenue, engagement, a blend?
What is the update cadence?
Is there editorial discretion?
What is excluded? Bulk sales, certain retailers, certain formats.
Is it regional?
How deep does the published ranking go? A top-200 chart tells you nothing about position 500.
The truncation problem
Most non-marketplace charts publish only a top N.
Below the cutoff there is no data, not a poor position.
A product dropping off a chart has moved from measured to unmeasured, and the size of the drop is unknown.
Analysis is therefore restricted to the top, and any claim about the broader field is unsupported.
This is a harder constraint than the noise problems in marketplace long tails, because there the data exists and is noisy; here it does not exist.
The general lesson
Every ranking system answers a specific question about a specific population using a specific method.
Find out what the question, the population and the method are before treating the number as a measurement of anything. For marketplace sales ranks that information is unpublished and must be inferred. For several of the systems in this article it is documented, which makes them easier to analyse honestly and does not make the number mean more than it does.
Reading a published methodology
Where a chart operator publishes their method — which several do, unlike marketplaces — read it properly, because it changes the analysis.
What counts as a unit? Streams, sales, downloads, and at what weighting.
What is excluded? Bulk purchases, certain retailers, certain territories, certain account types.
What is the reporting population? A complete census or a panel.
What is the period? A rolling window or a fixed week.
When does it refresh, and what is the lag?
What rules exist about eligibility and editorial discretion?
How often has the methodology changed, and are historical figures restated?
That last question is the one that catches people: a chart whose methodology changed mid-series has a discontinuity, and unlike marketplace ranks, here you can find out exactly when and why.