What metrics can be measured?
After the benchmarks are run on the system under test (SUT), a ratio for each of them is calculated using the run time on the SUT and a SPEC-determined reference time. From these ratios, the following metrics are calculated: • SPECmpiM_base2007: The geometric mean of thirteen normalized ratios – one for each benchmark – when the benchmarks are compiled with base tuning. • SPECmpiM_peak2007: The geometric mean of thirteen normalized ratios – one for each benchmark – when the benchmarks are compiled with peak tuning. • SPECmpiM_2007: A generic term for the measured result, whose value is the larger of the SPECmpiM_base2007 and SPECmpiM_peak2007. Larger data sets will be added later on, with the metrics • SPECmpiL_base2007 • SPECmpiL_peak2007 • SPECmpiX_base2007 • SPECmpiX_peak2007 In all cases, a higher score means better performance on the given workload.
The HPC2002 suite can be used to measure and calculate the following metrics: • SPECseis
A9: The benchmark suites can be used to measure and calculate the following metrics: SPEC OMPM2001 SPECompMpeak2001: The geometric mean of 11 normalized ratios (one for each benchmark) when compiled with “aggressive” optimization and possible code modification. SPECompMbase2001: The geometric mean of 11 normalized ratios (one for each benchmark) when compiled with “conservative” optimization (all benchmarks compiled with the same flags and no modifications to source code). SPECompM2001: greater of base & peak metric. SPEC OMPL2001 SPECompLpeak2001: The geometric mean of 11 normalized ratios (one for each benchmark) when compiled with “aggressive” optimization, and possible code modification. SPECompLbase2001: The geometric mean of 12 normalized ratios (one for each benchmark) when compiled with “conservative” optimization (all benchmarks compiled with the same lags and no modifications to source code). SPECompL2001: greater of base & peak metric. The ratio for each of the benchmarks is
