Application-motivated, holistic benchmarking of a full quantum computing stack

Daniel Mills, Seyon Sivarajah, Travis L. Scholten, Ross Duncan

Research output: Contribution to journalArticlepeer-review

8 Citations (Scopus)
18 Downloads (Pure)


Quantum computing systems need to be benchmarked in terms of practical tasks they would be expected to do. Here, we propose 3 "application-motivated" circuit classes for benchmarking: deep (relevant for state preparation in the variational quantum eigensolver algorithm), shallow (inspired by IQP-type circuits that might be useful for near-term quantum machine learning), and square (inspired by the quantum volume benchmark). We quantify the performance of a quantum computing system in running circuits from these classes using several figures of merit, all of which require exponential classical computing resources and a polynomial number of classical samples (bitstrings) from the system. We study how performance varies with the compilation strategy used and the device on which the circuit is run. Using systems made available by IBM Quantum, we examine their performance, showing that noise-aware compilation strategies may be beneficial, and that device connectivity and noise levels play a crucial role in the performance of the system according to our benchmarks.
Original languageEnglish
Article number415
Number of pages33
Publication statusPublished - 22 Mar 2021


  • quantum computing
  • quantum computers
  • QCVV
  • Quantum Characterization, Validation and Verification
  • holistic benchmarking
  • compilers
  • computation


Dive into the research topics of 'Application-motivated, holistic benchmarking of a full quantum computing stack'. Together they form a unique fingerprint.

Cite this