C-RNN-GAN: Continuous recurrent neural networks with adversarial training

In plain words

Most computer models of music treat notes as a fixed menu of symbols, such as "C4" or "D#4", and pick one from the menu at each step. This paper describes each musical event with four plain numbers instead: how long the note lasts, its pitch as a frequency, how loud it is, and how much time passes since the previous note started. Because the values are continuous, the model can in principle represent any length or loudness, and chords can be expressed as notes that start at the same moment.

The model uses two recurrent networks, which are networks that carry a memory from one step to the next. The generator writes a piece one event at a time. The discriminator reads a whole piece and guesses whether it was written by a person or by the generator. The generator is trained to fool the discriminator, and the discriminator is trained to catch it. Before this contest starts, the generator is first taught plain next-event prediction on real music, so it learns basic structure first.

The author trained the model on classical music in MIDI form. The generated pieces become more varied as training goes on, but the evaluation uses simple statistics about the notes, and no formal listening study was done.

TL;DR

• Task: generate sequences of continuous musical events (tone length, frequency, intensity and time since the previous tone) with a GAN trained on classical MIDI §1 §2.1 §3.
• Model: generator and discriminator are each two stacked LSTM layers with 350 units per cell; the discriminator is bidirectional and the generator is not §3.
• Training: six epochs of squared-error next-event pretraining, then adversarial training with feature matching and a freezing rule that pauses whichever side is too strong §3.
• Result (author-reported): adversarial training gives more variety in tones and intensity span than the maximum-likelihood baseline; the three-tones-per-cell variant gets a higher polyphony score §4 fig. 3a–c.
• No numeric results table exists in the paper; every comparison is a curve in Figure 3 fig. 3.

Problem

Most RNN music models use a symbolic representation, where each tone is one discrete symbol §1. Discrete symbols cannot be passed back through a network, so adversarial training of such models needs workarounds such as policy gradients §1. The paper asks whether a sequence model can instead work on fully continuous data for tone lengths, frequencies, intensities and timing, and whether adversarial training is viable for it §1 §5.

The data are MIDI "note on" events from 3697 classical music files by 160 composers §3.

Method

1. Event representation. Each event is four real values: tone length, frequency, intensity and the time since the previous tone §2.1. Overlapping or simultaneous tones are represented by zero time between them §2.1.
2. Multi-tone variant (C-RNN-GAN-3). Each LSTM cell outputs up to three tones at once. Each tone has its own four values, and a missing tone has zero intensity. The discriminator is changed to match §2.1.
3. Generator G. A two-layer LSTM with 350 units per layer §3. At each step the input is a random vector v, uniform in [0,1]^k, concatenated with the output of the previous step; k is the number of features per tone, which is 4 §2 §3.
4. Discriminator D. A two-layer bidirectional LSTM with 350 units per layer §3. The same learned linear map followed by a sigmoid is applied to every step's output §3. The per-step values are averaged into one decision for the whole sequence §3.
5. Adversarial losses. The generator and discriminator losses are given below. Here z is a sequence of uniform random vectors in [0,1]^k and x is a training sequence §2.

L_G = \frac{1}{m} \sum_{i=1}^{m} \log\left(1 - D(G(\boldsymbol{z}^{(i)}))\right) \qquad [\S 2]

L_D = \frac{1}{m} \sum_{i=1}^{m} \left[ -\log D(\boldsymbol{x}^{(i)}) - \log\left(1 - D(G(\boldsymbol{z}^{(i)}))\right) \right] \qquad [\S 2]

6. Pretraining. The model is trained for 6 epochs with backpropagation through time and mini-batch stochastic gradient descent, learning rate 0.1 §3. The objective is squared error on the next event in the training sequence §3. Sequences start short and get longer over the epochs, which the authors call curriculum learning §3.
7. Feature matching. For the adversarial phase the generator's loss is replaced by a feature-matching loss. It compares the discriminator's representation R of real and generated sequences, taken from the last layer before the final logistic layer §3.

\hat{L}_G = \frac{1}{m} \sum_{i=1}^{m} \left( R(\boldsymbol{x}^{(i)}) - R(G(\boldsymbol{z}^{(i)})) \right)^2 \qquad [\S 3]

8. Freezing. Updates of D are stopped whenever its training loss is below 70% of G's training loss. The paper says the same is done for G when G becomes too strong, without giving the rule §3.

# Pretraining, 6 epochs: squared error on the next event  [§3]
def pretraining_loss(generator, songs):
    # Random vector v, uniform in [0,1]^4, at every step  [§3]
    noise = uniform(0, 1, shape=(B, T, 4))  # (B, T, 4)
    # Previous event fed to each cell; source is ambiguous, see Pitfalls  [§3]
    previous = shift_right(songs)  # (B, T, 4)
    predicted = generator(noise, previous)  # (B, T, 4)
    return mean(square(predicted - songs))

# Adversarial phase: D scores real and generated sequences  [§2] [§3]
def discriminator_losses(d_real, d_fake):
    # d_real, d_fake: sequence-level decisions in (0, 1)  [§3]
    d_loss = mean(-log(d_real) - log(1 - d_fake))  # L_D  [§2]
    return d_loss

# Generator loss with feature matching replacing L_G  [§3]
def generator_loss(features_real, features_fake):
    # Match the representation R, averaged over the batch  [§3]
    return mean(square(features_real - features_fake))

# Freezing: pause D while it is too strong  [§3]
def freeze_discriminator(d_loss, g_loss):
    # D is frozen while its loss is below 70% of the generator's  [§3]
    return d_loss < 0.7 * g_loss

Hyperparameters

Results

Author-reported. Setup: classical-music MIDI, 3697 files from 160 composers §3. Statistics were computed per training epoch on generated music fig. 3. The paper has no numeric table, and hardware and number of runs are not stated.

• Adversarial training vs baseline §4 fig. 3a, 3b. The baseline is a generator-like network trained only to predict the next event §3. The adversarial model shows more variety in the number of unique tones and in tone span. The baseline's unique-tone count stays consistently lower, and its tone span follows that count more closely fig. 3a, 3b §4.
• Scale consistency fig. 3a, 3b. The authors say the baseline's scale consistency is similar to C-RNN-GAN's. Scale consistency for C-RNN-GAN levels off after about ten to fifteen epochs fig. 3a.
• Three tones per cell fig. 3c §4 §5. C-RNN-GAN-3 has a higher polyphony score than both C-RNN-GAN and the baseline. After about epoch 50 to 55 it also reaches substantially higher tone span, unique tones, intensity span and three-tone repetitions fig. 3c.
• Real music fig. 3d §4. Real music has an intensity span similar to the generated music and slightly higher scale consistency that varies more. Its polyphony score is similar to C-RNN-GAN-3's fig. 3d.
• Listening. The authors did not run a thorough listening study. Their own impression is that the feature-matching variant gives a better balance between structure and surprise than the other variants §4.1.

Paper vs code

The repository linked in the paper's footnote is the author's own (github.com/olofmogren/c-rnn-gan, commit 9edafc0) §3. It is TensorFlow 1.x code and has no LICENSE file.

• Hidden size. The paper uses 350 units §3. The code defaults to 100 for both G and D rnn_gan.py. The run scripts pass --model medium, which this commit does not define job.sh, so the 350 value cannot be confirmed from the code.
• Undefined flags. The run scripts and the README pass --model, --feed_previous and --bidirectional_d job.sh job4.sh. None of these is defined in rnn_gan.py. The code defines --disable_feed_previous and --unidirectional_d instead rnn_gan.py rnn_gan.py. The scripts therefore do not run as written with this commit; the code's defaults (feeding back and bidirectional) match the paper's description.
• Feature matching is off by default. The code default is off rnn_gan.py. The run scripts turn it on job.sh.
• Feature-matching loss. The paper averages the squared difference over the batch §3. The code sums it over the batch, all time steps and all features, using the bidirectional RNN outputs rather than a single vector rnn_gan.py. This changes the gradient scale, which the code then clips at norm 5.0 rnn_gan.py.
• Generator loss. The paper's L_G is log(1 − D(G(z))) §2. The code uses −log D(G(z)) rnn_gan.py. It uses this loss only when feature matching is off rnn_gan.py.
• Discriminator learning rate. The paper gives one learning rate, 0.1 §3. The code gives D half of it: d_lr_factor defaults to 0.5 rnn_gan.py rnn_gan.py.
• Pretraining input. The paper says the input to each cell is the random vector concatenated with the output of the previous time step, and it uses the same wording for pretraining §3. The code feeds the previous ground-truth event during pretraining rnn_gan.py.
• Generator input. The paper feeds the previous cell output §2. The code feeds a linear projection of the cell output, and it passes the concatenation through a ReLU layer of hidden-size width before the LSTM rnn_gan.py. The initial fed-back value is uniform in [0,1] rnn_gan.py.
• Freezing. The code's loop is run_epoch(session, m, loader, 'train', m.opt_d, m.opt_g) rnn_gan.py, but the function's parameters are eval_op_g, eval_op_d rnn_gan.py. So the variable op_g holds the discriminator update. With that swap, the 70% rule freezes D as the paper says rnn_gan.py. The code also freezes G on every batch where either loss is below 2.0, a rule the paper does not mention rnn_gan.py. The losses used are those of the previous batch, starting from 10.0 rnn_gan.py.
• Dropout. The paper does not mention dropout. The code applies dropout with keep probability 0.5 to the generator cells and to the discriminator inputs and cells during training rnn_gan.py rnn_gan.py rnn_gan.py.
• Gradient clipping and L2. Global-norm clipping at 5.0 and an L2 term of 0.1 are not in the paper rnn_gan.py rnn_gan.py.
• Feature scale. The paper says the data are normalized to 384 ticks per quarter note §3. The code converts timing to that resolution music_data_utils.py music_data_utils.py, but does not rescale the four features. They stay in their raw units: frequency in Hz, intensity as the MIDI velocity, and lengths and times in ticks music_data_utils.py music_data_utils.py.
• Random windows. Each batch takes a random window of songlength events from each song music_data_utils.py. The paper does not state this.
• Random input size. The code sets the random input size to the number of song features, which is 4 for one tone per cell, so k = 4 matches the paper music_data_utils.py rnn_gan.py.
• Unused flag. init_scale is defined but not used elsewhere in the code rnn_gan.py.
• Polyphony score. The paper defines it as how often at least two tones start at exactly the same time §3.1. Reading the loop in get_polyphony_score, each note-on event is added to a count once per update, so a two-note chord scores 0.5, not 1.0 midi_statistics.py. I read this from the code and did not run it.
• Repetition normalization. The paper divides real-music repetition counts by l_r / l_g fig. 3d. midi_statistics.py returns raw counts and has no such normalization midi_statistics.py.
• Scales. The paper says "standard scale" without naming the set §3.1. The code uses major, natural minor and harmonic minor, and ignores melodic minor midi_statistics.py.
• Licence. The repository has no LICENSE file. Some source headers say Apache-2.0, but that is not a repository license rnn_gan.py. No code is quoted here, and the code is cited by path and line only.
• Hyperparameter table. The TeX source has a commented-out table of learning rates and hidden units [paper source, \iffalse block]. It is not in the compiled paper, so this card does not use it.

Pitfalls

• Results are plots only. Every comparison is a curve over epochs in Figure 3 fig. 3. No numbers are given in the text, and the number of runs or seeds is not stated.
• Evaluation is heuristic. The statistics are simple counts such as tone span, unique tones and repetitions §3.1. The authors say a thorough listening study has not been done §4.1. The polyphony measure only counts tones with exactly the same start time, which the paper itself notes §3.1.
• Ambiguous input for pretraining. "The output at the previous time step" can mean the ground-truth event or the model's own output §3. The code uses the ground truth rnn_gan.py. Teacher forcing in pretraining and free-running generation in the adversarial phase make the two phases differ.
• Initial state not stated. The paper does not say what the first previous-event input is. The code uses uniform random values rnn_gan.py rnn_gan.py.
• Freezing needs the code. The paper gives only the 70% rule. The 2.0 threshold, the always-freeze-G rule and the use of previous-batch losses are in the code rnn_gan.py. An implementation that follows only the paper will train differently.
• Unnormalized inputs. Frequency, intensity and time values are in their raw units, while the random inputs are in [0,1] music_data_utils.py. Feature scaling is not stated in the paper, and an implementation must choose one.
• Dropout during sampling. The sampling code uses the same training graph, which is built with is_training=True rnn_gan.py. The dropout layers have no training switch, so I read them as still active when sampling. I did not run this to confirm it.
• Unstated sizes and schedules. Batch size, sequence length, number of epochs, stopping rule, weight initialization, dropout, clipping and the L2 coefficient are not stated in the paper §3. The code's values are listed in Hyperparameters.
• Training stops on a clock. The code exits after 1440 minutes by default, even before max_epoch rnn_gan.py rnn_gan.py.
• Cost. Hardware, training time and memory are not stated. The run scripts submit jobs to a GPU queue submit_job.sh.
• Dataset rights. The paper does not state the licences of the MIDI files. The repository's data/ directory holds downloaded MIDI and should not be redistributed.

Lineage

• Builds on: the generative adversarial network framework §1; LSTMs §2; feeding the previous output back as input, following Mikolov et al. (2010) and Eck and Schmidhuber (2002) §2; feature matching and freezing, following Salimans et al. (2016) §3. The paper cites these works without arXiv ids, so they are not linked here.
• Compared against: a maximum-likelihood baseline with the same generator structure §3; and the discrete-symbol adversarial music model of Yu et al. (2016), which the paper says needs policy gradients §1.
• Superseded by: not stated in this card's sources.