Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
This paper separates deterministic recurrent history from a high-dimensional conditional output distribution, comparing RNN–RBM, RTRBM, and RNN–NADE models on binary piano rolls and using the learned symbolic prior in transcription.
Frame-wise independent Bernoulli outputs cannot represent which notes should sound together. The paper addresses that conditional-output problem by using either an RBM or NADE for each next piano-roll frame while a recurrent state carries history across frames.
Model distinction #
- In an RTRBM, the previous conditional hidden mean participates directly in recurrence.
- In an RNN–RBM, a deterministic recurrent state emits time-varying RBM biases; stochastic RBM hidden states model the within-frame distribution.
- RNN–NADE replaces approximate RBM likelihood learning with a tractable autoregressive conditional distribution.
- The same symbolic sequence model can act as a prior during audio-to-piano-roll transcription.
Limitations worth preserving #
The reported samples often lacked meter and long-term structure. Short or slowly mixing Gibbs chains weakened conditional RBM training, while greedy transcription search limited inference. Dataset split conventions and transpositions also matter when comparing historical scores. These limitations support separating local polyphonic compatibility from long-range musical form and separating model quality from decoder/search quality.
Visual audit #
All eight pages were rendered and inspected at page scale; the model graphs, piano-roll samples, likelihood table, and transcription schematic were opened at full resolution. No source visual is retained. The GTK energy, recurrence, and symbolic-representation diagrams separately preserve the reusable causal structures without carrying over source-specific notation or historical result plots.