Summarize metrics with random deletion
You have a metric for which you have a result every second You can’t keep this granularity forever; it would be too big Standard solution: produce e.g. hourly logs with summaries e.g. min max mean p50 p99 My suggested alternative: just keep the original data points, but randomly delete some You can then run any aggregations over them when required
How does random deletion affect expected percentiles?
More by Jim
- Time is running out to catch COVID-19
- The inception bar: a new phishing method
- The hacker hype cycle
- Project C-43: the lost origins of asymmetric crypto
- How Hacker News stays interesting
- My parents are Flat-Earthers
- The dots do matter: how to scam a Gmail user
- The sorry state of OpenSSL usability
- I hate telephones
- The Three Ts of Time, Thought and Typing: measuring cost on the web
- Granddad died today
- Your syntax highlighter is wrong
Tagged . All content copyright James Fisher 2018. This post is not associated with my employer. Found an error? Edit this page.