Scenarios
A vendor's benchmark shows their product is four times faster. What
would you do — accept it as directionally right, or find out what the
comparison was run against, on what hardware, and who chose the
workload?
You're building the case for a decision you already believe in. What
would you do — assemble the strongest supporting evidence, or spend the
first hour looking specifically for the best argument against it?
Someone quotes a well-known industry statistic to close down a
debate. What would you do — accept it as common knowledge, or ask
where it comes from, given that "everyone knows it" is a claim about
circulation rather than evidence?
Takeaway
- Ask what the claim would predict if it were false. If the answer is
"nothing different," it isn't doing any work.
- Trace numbers to their source before repeating them. Every hop strips
a qualifier.
- Check what a metric actually measures, not just what it's called.
- Apply your most demanding standard to conclusions you like. Nobody
else will.
- Separate "this argument is well constructed" from "this argument is
correct." Fluency is a property of the speaker, not the world.