its common, very few in the industry actually do realistic/useful benchmarks, those who do never take averages… its never a stupid single number…
here is an example of good benchmark:
All times in nanoseconds:
Limit order placement
50%: 1160
90%: 1180
97%: 1220
99%: 1420
99.7%: 1900
99.9%: 2220
99.97%: 3440
99.99%: 4580
this is called a distribution of whatever you’re measuring… when you see 99% it means the worst number your benchmark observed in 100 attempts was 1420 but at 99.99% e.g. worst number after trying 10000 times was 4580 nanoseconds… thats 0.45 milliseconds.
Averages are the worst most misleading numbers, we never use them in our benchmarks, its only good for marketing people…