Getting the data and keeping it honest

• Where Rank Data Comes From

  Four routes, with different costs, coverage, legal positions and failure modes. Most operations end up with two of them and should understand both.

  Reference

• Scraping, Terms of Service and the Limits

  The legal position is unsettled and jurisdiction-dependent; the contractual position usually is not. What to weigh before collecting.

  Reference

• Product Identifiers and the Joins That Break

  ASIN, ISBN, GTIN, SKU and internal codes rarely map one to one. Getting the join wrong produces a dataset that looks right and tracks the wrong thing.

  Reference

• Storing Rank Data So It Stays Useful

  The dataset is small, the history is irreplaceable, and most of the value comes from context fields people do not capture at the start.

  Procedure

• Building Your Own Tracking, Realistically

  The collection is a weekend. The maintenance is indefinite, and it is what determines whether the dataset is trustworthy in year two.

  Analysis

• Gaps, Nulls and Silent Data Failures

  The characteristic failure in rank collection is not an error but a plausible wrong value. Four monitors that catch it, and the manual check that catches the rest.

  Procedure

6 notes