SPEC Invalidates Over 2600 Intel CPU Benchmark Results Due to Custom Compiler Optimizations

Published by

teaser
The Standard Performance Evaluation Corporation (SPEC) has found that Intel's oneAPI DPC++ compiler was optimized in a way that breached SPEC's guidelines, leading to the disqualification of over 2,600 SPEC CPU 2017 benchmark scores for Intel processors. The issue arose because the compiler was optimized using specific knowledge of SPEC's benchmarks, particularly for the 523.xalancbmk_r/623.xalancbmk_s tests, in a manner that was not widely applicable, contradicting SPEC's Rule 1.4 which advocates for general optimizations.

These disqualified results, mainly affecting Intel Xeon processors, particularly older versions, will remain in SPEC's database for archival purposes but will no longer be actively published. Investigations indicated that these targeted optimizations could lead to up to a 9% performance improvement in certain scenarios, including a 4% increase in SPECint speed metrics.

The issue was specific to Intel oneAPI versions 2022.0 through 2023.0, with no impact on versions before 2022.0 or after 2023.0. Notably, the fourth-generation Intel Xeon Sapphire Rapids processors were the most affected. Intel has corrected this by removing the contentious optimizations in the version 2023.2.3 of the compiler, which is intended for use with the newer fifth-generation Xeon Emerald Rapids processors.

SPEC, an authoritative entity founded in 1988 for benchmarking application performance, comprises members from academia, research, and the IT industry. The SPEC CPU 2017 benchmark suite, consisting of 43 tests in four categories, is designed to assess the CPU subsystem's performance, including both integer and floating-point operations and their concurrency capabilities.


7868768


Share this content
Twitter Facebook Reddit WhatsApp Email Print