Generation for arbitrary length

[SashaBurashnikova]

Hello,

Maybe you could help me with my problem)When i run your generation.py script with defaults parameters it generates correctly 16 bars output for any midi file. But when i tried to change the length from 1280 to length of my midi-file (11134) and 16 bars to 76 bars the model output the mistake. Do you know how to fix it? Thank you in advance!

[FATAL] model stuck, exiting ...
Traceback (most recent call last):
File "generate.py", line 299, in
temperature=config['generate']['temperature'],
ValueError: too many values to unpack (expected 3)

[SashaBurashnikova]

Or maybe you could show some other possibilities to generate output with an arbitrary length? I found in the text of paper that at inference step we could generate output of arbitrary length, but everything what i tried to change doesnt work (max events, max target bars).

Thank you in advance!

[slSeanWU]

Hi Sasha,

I think you shouldn't change the input length (1280) in the input file if you are using our pretrained checkpoint, since Transformers are very likely to fail if you feed input sequences that are longer than those seen during training.

You may just leave the length at 1280, and generate.py will take care of all operations needed to generate arbitrarily long sequences.

Hope this helps!

[SashaBurashnikova]

Thank you for the answer, but what to do if i want to generate the output with the same length as was my input file? For ex. i take as the input 3 minute midi file and want to get as output 3 minute file but with changed polyphony and rhythm?)

[SashaBurashnikova]

I mean my question is how to define the length of the sequence that will be generated?

Thank you in advance!

[slSeanWU]

Ok, I think now I know where it goes wrong. I will update the config file to fix this.
Thanks for raising this issue!

For now, you may try the following as a workaround:

• change max_bars and dec_seqlen in the config file to match your input piece
• On line 284, generate.py, set max_input_len=1024 (I think this will be safer than 1280)

Let me know if these work.

[SashaBurashnikova]

Thank you!I've tried to do what you suggest to change max_bars and dec_seqlen in the config and On line 284, generate.py, set max_input_len=1024, but still have the same error:

[FATAL] model stuck, exiting ...
Traceback (most recent call last):
File "generate.py", line 300, in
temperature=config['generate']['temperature'],
ValueError: too many values to unpack (expected 3)

If you will find another solution. please. let me know!)

[SashaBurashnikova]

Hello,

I'm very sorry for disturbing you again but am i correctly understand that you will correct the config that it would be possible to define the length of sequence that will be generated?

[slSeanWU]

Sure. Will look into this problem in the coming days.

[slSeanWU]

Hi Sasha,

I've pushed the new config, and it worked well on my side in generating long pieces.

Regarding your comments, it's not possible to "define" the length of generated sequences since it's part of the model's freedom. All we can do is to restrict the sequence length fed to the model.

Also, if the model fails within the sequence length it was trained on (1,280), I think most likely there is something wrong with your input, either:

• The music is "too different" from what MuseMorphose has seen during training, such that it fails to generalize to.
• The sequence format is gramatically wrong (e.g., the ordering of Note_Pitch, Note_Velocity, Note_Duraion tokens is messed up).

Above are the points you may check. Hope this helps!

[SashaBurashnikova]

Hello,

Thank you!
Am i correctly understand you the i could change these parameters of config for generation part: max_bars, dec_seqlen, max_input_dec_seqlen - to change the maximum possible length of generated output?
And the generated length will be arbitrary but no longer the installed max_bars, dec_seqlen?

[SashaBurashnikova]

Sorry, maybe i don't exactly understand what do you mean, Because I see on your demos that you generated full song with modifications and the generated length for both is almost the same and are 4 min 1 sec and 4 min 3 sec? Do you use the model trained on sequences with length 1280?
How did you do it?
https://slseanwu.github.io/site-musemorphose/

Sincerely yours,
Aleksandra

[slSeanWU]

Actually, the length of the generated sequence is not completely tied to the wall-clock duration of a song. Higher attribute classes will lead to more notes played in the same amount of time, hence a longer sequence.

You may think of max_bars, dec_seqlen as restrictions applied on your input song. And max_input_dec_seqlen ensures the decoder does not see a sequence longer than what it has seen during training (i.e., crop a generated sequence to the most recent max_input_dec_seqlen events).

[MaximeHoude]

Higher attribute classes will lead to more notes played in the same amount of time, hence a longer sequence.

Regarding your comments, it's not possible to "define" the length of generated sequences since it's part of the model's freedom.

So in other words, it won't handle properly a 4 minutes "complex" piano score (with "a lot" of notes).

Could it be a solution to reset the time indices once it exceeds 1280, so the trained-model continue from where it left off?
Thanks!

[slSeanWU]

@Chopders

Actually, the current inference script will take care of these "complex" cases.
The context will be truncated (to 512, if I remember correctly) once it reaches max training length.

Thanks!

[dedededefo]

嗨萨沙，

我认为如果您使用我们的预训练检查点，您不应该更改输入文件中的输入长度 (1280)，因为如果您输入的输入序列比训练期间看到的序列长，Transformers 很可能会失败。

您可以将长度保留为 1280，generate.py并将处理生成任意长序列所需的所有操作。

希望这可以帮助！

Hello, do you know how to train the model with your own data? Remi in folder pickles_ vocab.pkl、train_ pieces.pkl、test_ pieces.pkl、val_ How to get pieces.pkl？ Thanks!

[dedededefo]

更高的属性等级将导致在相同的时间内演奏更多的音符，因此序列更长。

关于您的评论，不可能“定义”生成序列的长度，因为它是模型自由的一部分。

所以换句话说，它无法正确处理 4 分钟的“复杂”钢琴乐谱（带有“很多”音符）。

一旦超过 1280，是否可以重置时间索引，以便训练模型从停止的地方继续？ 谢谢！

Hello, do you know how to train the model with your own data? How to get remi_ vocab.pkl、train_ pieces.pkl、test_ pieces.pkl、val_pieces.pkl in folder pickles? Thanks!

[dedededefo]

Or maybe you could show some other possibilities to generate output with an arbitrary length? I found in the text of paper that at inference step we could generate output of arbitrary length, but everything what i tried to change doesnt work (max events, max target bars).

Thank you in advance!

Hi！What is Remi_ vocab.pkl、train_ pieces.pkl、test_ pieces.pkl、val_ pieces.pkl in folder pickles? How to get them？ Thanks

[dedededefo]

Actually, the length of the generated sequence is not completely tied to the wall-clock duration of a song. Higher attribute classes will lead to more notes played in the same amount of time, hence a longer sequence.

You may think of max_bars, dec_seqlen as restrictions applied on your input song. And max_input_dec_seqlen ensures the decoder does not see a sequence longer than what it has seen during training (i.e., crop a generated sequence to the most recent max_input_dec_seqlen events).

Hi！What is Remi_ vocab.pkl、train_ pieces.pkl、test_ pieces.pkl、val_ pieces.pkl in folder pickles? How to get them？ Thanks