Comments on: Say What?

By: Kyle

Kyle — Thu, 19 Apr 2007 22:22:35 +0000

Parand, that was exactly my point. We can compare two sites — but not by the metrics most people use. We can compare them based on the metrics (subjective metrics) I was describing. Google is “fast” because people perceive it to be fast. Not because of the technology it is built on.

I’m not copping out, I’m simply throwing out meaningless data. To me, any kind of request/second measurement based on technology is meaningless. Because it’s irrelevant. It doesn’t describe the data we’re trying to compare (perceptive speed).

It’s like saying “ferraris are faster than kias” then racing the two with the ferrari hauling a trailer, and the Kia stripped down to bare mechanics. While the numbers might suggest the ferrari is faster, the truth is quite different.

By: Parand Darugar

Parand Darugar — Thu, 19 Apr 2007 21:07:39 +0000

“due to the number of changing variables, comparing one entire site to another is a bit ridiculous.”

This is the fundamental point we disagree on – you can compare the performance of an entire site to another, and you absolutely should. Speed is a feature. Altavista got its early gains precisely because it was lighting fast. Google not only prided itself on being relevant, but also very fast.

At every occasion possible and with every meaningful measurement possible you should compare performance. I think it’s a cop-out to say it’s complicated and there are many variables, therefore we can’t make any statements.

You’re right – “X is faster than Y” is a gross approximation, but sometimes gross approximations are good. “Ferraris are faster than Kias”. That would be a pretty accurate statement.

By: Kyle

Kyle — Thu, 19 Apr 2007 19:10:17 +0000

Parand,

I believe you misinterpreted my post in quite a number of ways. I do not dislike statistics; I love them. Remember, I’m a Engineer and deal with statistics on a day-to-day basis.

There are a ton of meaningful statistics you can measure about your application, but what I’m saying is that we cannot provide any meaningful results out of them. They are are all relative measures of performance are great ways to measure gains (i.e. 10x faster than version 1.0), and great ways to measure bottlenecks. However, due to the number of changing variables, comparing one entire site to another is a bit ridiculous.

The same goes for comparing two frameworks, or languages.

Do you often see statements like: “Suburus are faster than VWs”? You don’t because of the number of variables involved. Which cars? What kind of fast? Etc. However, you see almost every day statements like “Django is faster than Rails” Which versions? What kind of fast? Those questions are never answered, making their statistics non-meaningful.