Independent Cross-Stack Benchmark Evaluates Commercial Quantum Error Management on 156-Qubit IBM Heron Hardware
An independent benchmark on IBM Heron hardware evaluated commercial quantum error management tools. …
An independent benchmarking study published on arXiv (arXiv:2608.05202) provides a protocol-level evaluation of commercial quantum error suppression and error mitigation software stacks. Conducted by researchers from The Catholic University of America, University of Deusto, and Universidad de los Andes, the paper compares native IBM Qiskit Runtime primitives against third-party managed pipelines in the Qiskit Functions Catalog—specifically Q-CTRL Performance Management and Qedma QESEM. All experiments were executed on ibm_pittsburgh, a 156-qubit IBM Quantum Heron r3 superconducting processor.
The benchmark evaluates two execution models: sampling tasks (bitstring counts) via Sampler interfaces, and observable expectation values via Estimator interfaces. To ensure like-for-like comparisons, IBM and Q-CTRL configurations operated under matched shot budgets (215 = 32,768 shots per instance), while Qedma QESEM was configured through its native precision-and-time-budgeted interface (0.1 precision target, 600-second QPU time cap). Abstract workloads were submitted across all providers within identical hardware calibration windows.
In the Sampler Track, tested across Bernstein–Vazirani (25–75 qubits), Quantum Phase Estimation (10–30 counting qubits), GHZ-state preparation (25–50 qubits), and Randomized Mirror Circuits (25–100 qubits), Q-CTRL’s autonomous suppression pipeline delivered the highest exact-output success probabilities across all 12 test cases. On 30-qubit Quantum Phase Estimation (QPE), where raw and measurement-twirled IBM runs yielded zero exact-match shots out of 32,768 (0.00% success), Q-CTRL maintained a 12.69% exact success probability. On 100-qubit Randomized Mirror Circuits, Q-CTRL retained a 76.45% exact 100-bit target string success rate, compared to 9.14% on IBM raw execution.
[ Estimator Track Accuracy & QPU Overhead Benchmarks (TFIM 25–75 Qubits) ] | ||
Software Stack Configuration | Error Metrics vs. MPS Exact Reference | Error Reduction & QPU Runtime |
IBM Raw Execution (SamplerV2 / EstimatorV2 Level 0) | • Magnetization MAE (mX): 0.1484 • Correlator MAE (cZZ): 0.0282 • Overall MAE: 0.0883 | • 1.00× (Baseline) • Reported QPU Time: ~17.8 seconds per job |
IBM TREX + Twirling (Resilience Level 1 + Gate Twirling) | • Magnetization MAE (mX): 0.1026 • Correlator MAE (cZZ): 0.0587 • Overall MAE: 0.0807 | • 1.09× Error Reduction • Reported QPU Time: ~28.0–37.8 seconds per job |
Q-CTRL Performance Management (Autonomous Suppression Pipeline) | • Magnetization MAE (mX): 0.0376 • Correlator MAE (cZZ): 0.0194 • Overall MAE: 0.0285 | • 3.10× Error Reduction • Reported QPU Time: ~28.0 seconds per job |
Qedma QESEM (Characterization & Unbiased Mitigation) | • Magnetization MAE (mX): 0.0278 • Correlator MAE (cZZ): 0.0097 • Overall MAE: 0.0188 | • 4.70× Error Reduction • Reported QPU Time: ~211–311 seconds per job |
In the Estimator Track, evaluated on an eight-layer Transverse-Field Ising Model (TFIM) circuit at 25, 50, and 75 qubits against an exact Matrix Product State (MPS) reference, Q-CTRL and Qedma QESEM reduced aggregate mean absolute error (MAE) by factors of 3.10× and 4.70× over raw execution, respectively. However, the two third-party stacks demonstrated distinct resource-accuracy profiles: Q-CTRL achieved its error reduction using 28 seconds of QPU time per job (matching IBM baseline runtime), whereas Qedma QESEM achieved the highest precision by consuming 211 to 311 seconds of QPU time per job (7.5× to 11.1× the runtime of Q-CTRL).
Review the research preprint on arXiv (Quantum Error Management Benchmark) here and explore cloud primitive specifications at IBM Qiskit Runtime here and here.
September 23, 2026
Related Intelligence
Test of USydney article with HTML.
EPFL Demonstrates Millimeter-Wave-to-Optical Transducer with 67% Internal Efficiency
November 10, 2026
Why Quantum Hardware Needs Its Own Threat Model
November 9, 2026
Physics Reports Analysis Evaluates Emergence of Hybrid Semiconductor–Superconducting Qubit Architectures
November 9, 2026
DLR Develops Scalable Unitary Data Encoding Beating Exponential State-Preparation Depth
November 9, 2026
IonQ and Classiq Outline Architectural Trade-Offs in Fault-Tolerant Resource Estimation
November 9, 2026