Does the choice of reference machine affect the metrics?
A19: No. SPECjvm98 metrics are calculated using the geometric mean, so relative rankings of different systems are independent of the reference machine. Q20: Why did you choose bytecode execution times instead of JITted execution times as reference. A20: This issue was debated, but we based our decision on two principal factors: (1) some platforms do not use JIT due to its larger memory footprint, and (2) JITted execution times could lead to a SPEC ratio of less than 1, a score that SPEC believes discourages people from reporting results.
Related Questions
- Why does SPEC use a reference machine for determining performance metrics? What machine is used for SPEC CPU2000 benchmark suites?
- Why does SPEC use a reference machine for determining performance metrics? What machine is used for SPEC OMP2001 benchmark suites?
- Does the choice of reference machine affect the metrics?