What are the rules regarding fair published comparisons of TPC-D results? There are two items regarding comparability of TPC-D results, as documented in Clause 5.
• The TPC discourages comparison of TPC-D benchmark results measured against databases of different sizes (i.e., scale factors). Database performance and capabilities, as well as price-performance ratios, may not scale proportionally with changes in database sizes; therefore, such comparisons may be misleading. Any printed communication which compares results measured against databases of different sizes must carry a specific disclaimer which states the TPC’s discouragement of such comparisons. • Given that the database scale factors are the same, any TPC-D result is comparable to any other TPC-D result regardless of the number of query streams used. By policy of the TPC, published comparisons of any TPC benchmark results must include all metrics for that particular benchmark. Therefore, the comparison of TPC-D results for two or more systems must include the power metric, the throughput metric, and the price-performance metric.
Related Questions
- What are the rules regarding fair published comparisons of TPC-D results? There are two items regarding comparability of TPC-D results, as documented in Clause 5.
- The language rules given in GURPS Basic Set give unrealistic results for high-IQ characters. Are there any better language rules for GURPS?
- Can SPEC MPI2007 results be published outside of the SPEC web site? Do the rules still apply?
