We've spent our week of introspection asking hard questions of ourselves and each other. We're ready to share what we've learned.Links Referenced in the Vide...
Then they need to preface every video with that. “We ran all these benchmarks and testing using hundred-thousand-dollar equipment and are presenting this as a factual review, but please keep in mind this is all entertainment and not educational and the numbers are made up so just disregard this whole video.”
I would contend that most of the YouTube tech channels, even the ones with great reputations for quality such as Hardware Unboxed and Gamers Nexus are barely educational as well.
While they can point to fewer mistakes made as a result of their methodologies, I don’t believe there is any real scientific value in the conclusions they reach from testing that is far too limited with far too small a sample size. They can paint broad recommendations - product A should be better than product B because our testing showed 20% better numbers.
But when the metric variances between products are small, none of the testing methodologies can really tell you which of the products will work better in your system. They haven’t tested enough of them, and in enough situations to have a clue. And I think any of them claiming that there is inherent value to their methods are really just defending their product which is the video they’re getting eyeballs in front of so they can make money from advertising.
Yes, if you buy product X tested on whatever YouTube channel, your mileage may vary.
However, LTT had several graphs where they showed egregious performance increases in newer models of graphics cards compared to older ones. Somebody drawing a conclusion just from the relative performance increase of over 200%, even if the numbers would’ve been slightly off otherwise, will be more likely to make a bad decision based on the relative (non-existent) insane performance increase.
LTT is entertainment, not educational.
Then they need to preface every video with that. “We ran all these benchmarks and testing using hundred-thousand-dollar equipment and are presenting this as a factual review, but please keep in mind this is all entertainment and not educational and the numbers are made up so just disregard this whole video.”
I would contend that most of the YouTube tech channels, even the ones with great reputations for quality such as Hardware Unboxed and Gamers Nexus are barely educational as well.
While they can point to fewer mistakes made as a result of their methodologies, I don’t believe there is any real scientific value in the conclusions they reach from testing that is far too limited with far too small a sample size. They can paint broad recommendations - product A should be better than product B because our testing showed 20% better numbers.
But when the metric variances between products are small, none of the testing methodologies can really tell you which of the products will work better in your system. They haven’t tested enough of them, and in enough situations to have a clue. And I think any of them claiming that there is inherent value to their methods are really just defending their product which is the video they’re getting eyeballs in front of so they can make money from advertising.
Yes, if you buy product X tested on whatever YouTube channel, your mileage may vary.
However, LTT had several graphs where they showed egregious performance increases in newer models of graphics cards compared to older ones. Somebody drawing a conclusion just from the relative performance increase of over 200%, even if the numbers would’ve been slightly off otherwise, will be more likely to make a bad decision based on the relative (non-existent) insane performance increase.