I hear this type of question often when I am talking about quantum computing. It is fascinating because the approach to the answer is not apparent. In most cases, it depends on the customer’s situation — its challenges, needs and goals.
The same question can be heard during conferences. There are some tries of providing the results during sessions. I saw many times comparison to the classical approach. The tricky part was that in the discussion came out that it was a brute-force approach. It is not something you would like to see because this is not a state-of-art approach that you would use in the project.
I understand those people. I had the same need when I started my journey with quantum computing — let’s compare it to something I know.
Do you think it’s a good approach?
How should we benchmark quantum solutions?
The answer is more complex — it depends on your situation.
Let’s not compare it at all
Do I need to compare it? Imagine that you are solving a new challenge for your organisation with quantum computing. Before starting the project, you have defined expectations:
- What quality should the results have?
- How fast should your solution be?
- What is the cost?
or any other goal that is important for your organisation. During development, you deliver quantum solutions that pass all your requirements.
Should you feel content?
Do you need to compare it with the classical one?
For me, this is a perfect situation where you do not need to compare the classical solution.
Comparison is simple — let’s do that
Another situation occurs when you already have a classical solution for your problem. You are probably investigating quantum computing, and here is an excellent opportunity to benchmark both options.
In the case of a solution that uses a classical approach, your model already has multiple iterations. It is adjusted to your needs and tuned to provide the best solutions. In such a situation, I trust in the results (in most cases).
There are two risks:
- You didn’t invest enough time in developing a quantum solution — if you do not have enough knowledge, transformation to quantum representation may not be the best. Here we are spotting the same issue as described at the beginning. It will not be the state-of-the are the solution
- Accurate problem size — this global challenge for quantum computing. Current devices are still a bit narrow in the scope of cases they can support and the size of the problem they can address. Running too small a problem will not generate value compared to classical applications. Selecting too big a challenge will cause failing calculation.
As you can see here, we have all the data, but we still need to be careful with our approach.
Should I always check which solution is the best?
It depends — as always, we must include business requirements. In most cases, adding a new model or approach to comparison is connected with cost. It can be money, time, resources or similar.
You should decide if this is worth doing. Imagine that you are optimising the schedule of people working in your Objectivity. Right now we have about 1000 people in the organisation. Then let’s do the same for security workers at airports — here we have hundreds of thousands of people employed. For both organisations, 1% better solutions mean something different. The bigger organisation will have a higher demand for getting the best result.
We can use the same approach for the size of the problem — the more significant problem we are optimising, the bigger the desire to get the best results. It is natural. Please remember to stop and check if further improvement is needed and if a better solution will generate additional value for the organisation.
Great article and advice Michal!