Comments on: Itâ€™s Time for a Decent Mac Benchmark

By: imajoebob

imajoebob — Mon, 11 Jun 2007 04:38:36 +0000

I’ve noticed that the speed of your hard drive seems to influence all the scores, not just the disk rating. Makes me wonder just how discrete the individual tests are.

By: Tutor

Tutor — Sun, 03 Jun 2007 13:43:05 +0000

You are right about Xbench. It doesn’t even produce the same results on the same machine – useless.

I favor timing real applications these days, too.

By: Jaysen

Jaysen — Fri, 01 Jun 2007 16:37:07 +0000

Someone should note teh max char limit on the comments…

What I was saying is that the 128bit

By: Jaysen

Jaysen — Fri, 01 Jun 2007 15:21:27 +0000

The problems with benchmarks are the type of calculations, the programmers “method” of performing calculations and the differences in hardware architecture. If we take a benchmark test that performs “large math” we may need different routines for each architecture (PPC, Intel Single/Dual/Quad/SMP) to optimize performance thereby reflect true possible speed. problem is that few software packages use separate routine optimizations (other than compiler) to maximize performance between architectures. So if we build a benchmark utility that is generic we penalize one or more architectures. On the other hand if we truly optimize then we are not comparing performance in a “real world” manner.

An example is dual core vs SMP. SMP is 2 chips, 2 sub systems. Dual core tries to emulate this but still gets bogged down in bus IO. So although we can use a similar function for the raw math the IO should be optimized for a dual core system. This would cause the SMP system to appear to not have a large benefit as the higher sustainable IO would never be measured. If we optimize for SMP then we penalize dual core by overloading the IO buss.

Also the “speed” rating of chips is not really an indicator of performance. Although a faster chip can cycle through calculations faster there are other factors like data size (16 vs 32 vs 64 vs 128) and bus speeds that have a larger impact. take a look at the initial offerings of almost any chip generation and you will find a speed overlap with the previous generation.

For those of you who remember the DEC “Alpha” series chips, they were 128bit

By: Matt J

Matt J — Fri, 01 Jun 2007 15:06:10 +0000

Would timing to completion of some kind of Automator workflow be an appropriate indicator?

By: Damien

Damien — Fri, 01 Jun 2007 13:56:18 +0000

I’d love to see someone create a series of standard tests that could be done by anyone and to collect them from different machines, so that we could really compare e.g. a PowerMac G4/533 (DA) against a 2ghz iMac (an upgrade I’m contemplating). Any suggestions on how to do the tests? I’d suggest using freely available tools (inc shareware / trialware), standardized files that can be loaded, and some Applescripts to launch & time the tests.