Variational quantum circuit

Overview

A variational quantum circuit (VQC) is a quantum circuit whose gate angles are trained by a classical optimiser, much as neural-network weights are trained. The input is written into the qubits, a fixed pattern of entangling gates and trainable rotations follows, and the output is read as expectation values of measurements 1907.00397 §4.1. The appeal is the small number of trainable settings, and the circuit depth must stay short because current noisy hardware makes errors grow with the number of gates 1907.00397 §1 1907.00397 §6.6.

The paper credits the circuit approach to Mitarai et al. (2018) 1907.00397 §1 1907.00397 §4. That work has no card here, so nothing from it is described on this page.

Variants

Only one variant has a card. Its design choices are listed so that a later card can be compared against them.

Bit-encoded circuit Q-network (Chen et al., 2019)

The state index is written in binary, one bit per qubit. A bit set to 1 is turned into the |1⟩ state with π rotations. Each layer applies a chain of CNOT gates between neighbouring qubits, then one general single-qubit rotation with three trainable angles on every qubit. The paper uses two layers 1907.00397 §4.1 1907.00397 fig. 5 1907.00397 eq. 15. The output for each action is the Pauli-Z expectation of one qubit plus a trainable bias, and the chosen action is the index of the largest output 1907.00397 §6.1. The circuit serves as the Q-function inside deep Q-learning 1907.00397 §4 1907.00397 alg. 1. Card: Variational Quantum Circuits for Deep Reinforcement Learning.

Reported results, all from simulation unless stated:

• Frozen lake uses 28 trainable parameters where a table has 64 entries, and the authors report a total reward of 0.9 after 198 episodes 1907.00397 §6.4 1907.00397 §6.6.
• Cognitive radio with 2 to 5 channels reaches a total reward of 100 in simulation 1907.00397 §6.4 1907.00397 fig. 8.
• Two short hardware runs, 7 and 5 episodes, are reported with no variance 1907.00397 §6.5 1907.00397 tab. 3 1907.00397 tab. 4.

When to use which

With one card there is nothing to compare against, so the table gives the constraints under which the paper's evidence applies.

Implementation sketch

The core is the read-out from a state to one value per action. The learning loop around it is the one described on deep-q-learning.

# Circuit read-out for one state  [arxiv:1907.00397 §4.1] [fig. 5]
# Input: state index, angles (2, n, 3), bias (n,); Output: n action values
def circuit_q_values(state_index, angles, bias):
    # Write the index in binary, one bit per qubit  [§4.1]
    bits = binary_digits(state_index, width=n_qubits)
    # Bit 1: pi rotations about x, then z  [eq. 7] [eq. 8]
    for qubit in range(n_qubits):
        if bits[qubit] == 1:
            rotate_x(pi, qubit)
            rotate_z(pi, qubit)
    # Two layers: CNOT chain, then one rotation per qubit  [eq. 15]
    for layer in range(2):
        for qubit in range(n_qubits - 1):
            cnot(control=qubit, target=qubit + 1)
        for qubit in range(n_qubits):
            general_rotation(angles[layer, qubit])
    # Pauli-Z expectation per qubit, plus a trainable bias  [§6.1]
    z = [expect_z(qubit) for qubit in range(n_qubits)]
    return [z[i] + bias[i] for i in range(n_qubits)]

Contradictions

• Target parameters. Algorithm 1 computes the target value from the current parameters, while the text says a separate target network is used 1907.00397 alg. 1 1907.00397 §2.3 1907.00397 §4.
• Convergence, noisy radio runs. The authors give an optimum at about 110 iterations 1907.00397 §6.4, and elsewhere "very close to 100 in fewer than 10 iterations" 1907.00397 §6.6.
• Convergence, noiseless radio runs. The caption of the reward figure says "several iterations" for 2 and 5 channels, while the text says about 100 1907.00397 fig. 8 1907.00397 §6.4.

Pitfalls

• The released code does not match the paper in several places: the radio code uses 4 qubits for every channel count, and its exploration rule and target sync differ from the text. See the card's Paper vs code section.
• Gamma is not stated in the paper. The released code uses 0.999, so the reported numbers are tied to that value. The frozen-lake episode length is also not stated.
• Weight initialisation is not stated in the paper, and the bias starting value is not stated either.
• The radio code reseeds its random generator from the clock, so its runs are not reproducible from the repository. It also uses an alias that NumPy 1.24 removed, so it will not run on current NumPy without changes.
• Parameter counts are not performance. The 28-parameter figure is a comparison with a table, and the radio neural-network count comes from a cited work that was not checked 1907.00397 §6.6.

Papers

• Variational Quantum Circuits for Deep Reinforcement Learning: a bit-encoded, two-layer circuit as the Q-function of deep Q-learning, tested on frozen lake and cognitive-radio channel selection in simulation.