Quantum Benchmarking Initiative History, Types & Challenges
QBI Quantum Benchmarking Initiative
DARPA's Quantum Benchmarking Initiative (QBI) is crucial to assessing quantum computing technology's development and potential. Its main goal is to determine if any quantum computing approach can achieve “utility-scale” functioning by 2033, where computational value exceeds cost. This initiative is crucial to turning quantum computing from experimental and theoretical to practical and industrial.
Stages of Operation
Every effective performer effort will go through the QBI's three critical stages:
A: Concept Description (6mo)
Businesses present detailed technological plans to demonstrate utility-scale quantum system development.
This level describes a utility-scale quantum computer concept with a good possibility of becoming a reality soon.
Each business can earn $1 million for Stage A. Businesses must answer questions about how they will build their quantum computer, why they think they can, how it will change the world, and their goals for the year. IonQ will analyze DARPA problem sets and use cases for large-scale machines to define utility-scale performance.
Stage B: 12-month R&D Plan
Stage B companies will review their research and development goals and determine how they will measure progress.
Businesses must now design a research and development strategy to construct a utility-scale quantum computer, including risks, risk reduction measures, and prototypes.
C: Verification & Validation (varying, 36 months)
Businesses who pass Stage B will build hardware and submit it to DARPA's rigorous independent verification and validation in Stage C. DARPA has assembled a trained test and evaluation team to leverage federal and state test facilities for these assessments.
Stage C aims to determine if a company's technology is effective and if the proposed system can be developed and run as intended for real-world deployment.
Quantum Benchmarking Architecture
Quantum benchmarking evaluates quantum computer hardware, software, and algorithms. The architecture incorporates multi-layered performance assessment at quantum stack levels:
Component-level: This layer assesses qubit and gate performance using coherence times and fidelity.
System-level: Examines the quantum processor as a whole, focusing on how errors spread across qubits.
Software-level: This layer assesses the quantum compiler and error-reduction methods.
Application-level: The highest level addresses real-world concerns from optimization, machine learning, and quantum chemistry, making benchmarks valuable.
Quantum Benchmark Types
Quantum benchmarks are categorized by complexity and measurement:
The main purpose of low-level benchmarks (diagnostics) is to identify hardware issues. Examples include Quantum Process Tomography, which describes a gate or process, and Randomized Benchmarking (RB), which determines quantum gate error rates.
Quantum computer volumetric benchmarks give a single number for their capability. Well-known examples include Quantum Volume (QV), which measures the largest “square” circuit (equal number of qubits and circuit depth) a machine can operate with satisfactory fidelity.
Application-Driven Benchmarks: These benchmarks assess a quantum computer's performance on practical workloads. They are crucial for assessing a machine's practicality. Training quantum machine learning models, addressing optimization problems, and modeling molecular behavior are examples of benchmarks.
Quantum Supremacy/Advantage Benchmarks: Like Random Circuit Sampling (RCS), these benchmarks prove that a quantum computer can compute faster than the best classical supercomputer.
QBI features
The QBI highlights key features:
Utility-Scale: By 2033, the goal is to build and validate “utility-scale” quantum computing, where advantages outweigh drawbacks.
Diverse Qubit Technologies: DARPA chose photonic, trapped ion, superconducting, and neutral atom qubit companies. Publicly traded companies include Atom Computing, Diraq, IBM, IonQ, Quantinuum, QuEra, and Xanadu.
QBI is meant to assess the commercial quantum landscape and support all possible paths to breakthrough quantum computing, not compete among performers.
Independent Verification and Validation (IV&V): An unbiased third party must verify and validate performers' paths to a utility-scale quantum computer.
Government Communication: QBI will inform U.S. government agencies interested in using or buying this technology of its successful verification and certification.
Partnerships: Quantinuum's Stage A effort includes Microsoft and NVIDIA, building on their longstanding partnerships to achieve commercially scalable quantum computing.
Advantages
Quantum benchmarking, especially QBI, has many benefits:
Standardisation: Benchmarks make comparing quantum computers and tracking their progress easy.
They promote quantum computing transparency by giving unbiased, verified performance statistics.
For utility-scale quantum computing, benchmarks let researchers and developers focus on the most relevant areas for development.
Reality Check: The QBI dispels myths and controls hype in the quantum computing business to maintain investor trust.
Inspiration for Others: The effort may inspire other quantum technology-focused countries like India to establish quantum benchmarking projects to generate separate development paths.
Disadvantages
Quantum benchmarking has drawbacks:
Manipulation: Benchmarks can be “gamed” to obtain certain results, resulting in erroneous quantum system comparisons.
Limited Scope: A quantum computer that performs well on one benchmark may not perform well on another.
Complexity: Creating robust and fair benchmarks for many quantum designs is tough.
Hype and Misrepresentation: Businesses that lie to influence stock markets and raise money sometimes fuel quantum computing hype and conjecture. “Quantum advantage” claims are often rejected. Using quantum AI hype to earn quick money, some businesses have been accused of generating “vapourware”.
Applications
Quantum benchmarking evaluates numerous quantum computing applications rather than the technology itself:
Quantum chemistry models molecular and material properties to expedite drug discovery and material design.
Machine Learning: Processing complex datasets to improve AI models.
Supply chain management, finance, and logistics optimization: Complex optimization problems.
Materials science: Finding new materials with desirable properties, such superconductors or battery catalysts.
These applications are expected to transform healthcare, banking, agriculture, energy, and the military.
Challenges
Quantum benchmarking has many issues:
Fair comparisons are difficult without an industry-wide quantum computer performance standard.
Benchmarking is challenging with superconducting, trapped-ion, neutral-atom, and photonic quantum computing systems' merits and cons.
Noisy, error-prone quantum computers can impair benchmark results. To consistently account for these mistakes is difficult.
Scalability: Small quantum computer benchmarks are difficult to apply to complex solutions.
Quantum technology is new and complex, making it impossible to distinguish fact from fiction.
Investor Trust: Hype and disinformation can deter investors and slow field progress.
History
QBI debuted in July 2024. Underexplored Systems for Utility-Scale Quantum Computing (US2QC) is expanding at DARPA. US2QC, which involves PsiQuantum and Microsoft, is in its last stage and attempts to prove commercial quantum computers are possible.
QBI is linked to Quantum Benchmarking (QB), a DARPA project to determine the “yardstick for impact”—what a fully functional quantum computer can do that a normal computer cannot. The QBI combines US2QC's hardware-validation expertise with QB's software and application focus.
DARPA sponsored a proposers day in September 2024 for companies seeking money and objective Independent Verification and Validation to construct a commercially functional quantum computer by 2033. In April 2025, DARPA announced that 18 companies had been selected for Stage A of the QBI, 15 of which were public and three were negotiating contracts.