Repeat inputs during prediction

[sudongwang-upc]

Why repeat here? The results generated after repeat (i.e. the parameters of the distribution) should be the same, right? What is the significance of this?

lag-llama/lag_llama/gluon/lightning_module.py

Lines 221 to 226 in 9965e30

	repeated_past_target = past_target.repeat_interleave(
	self.model.num_parallel_samples, 0
	) # (bsz* self.model.num_parallel_samples, model.context_length+max(model.lags_seq))
	repeated_past_observed_values = past_observed_values.repeat_interleave(
	self.model.num_parallel_samples, 0
	) # (bsz* self.model.num_parallel_samples, model.context_length+max(model.lags_seq))

[ashok-arjun]

When sampling, we require that we generate self.model.num_parallel_samples at every sampling iteration. Hence, we repeat all the tensors that many times.

[sudongwang-upc]

When sampling, we require that we generate self.model.num_parallel_samples at every sampling iteration. Hence, we repeat all the tensors that many times.

Thank you for your response to this issue. Let me elaborate on my question again.

1. The model output obtained from the same input should be the same.
2. In lag llama, the distribution parameters of the output obtained from multiple identical inputs after repeating all the tensors should be the same, right?
3. If the same distribution samples n times and n distributions with the same parameters are sampled once per distribution, the results should be consistent, right?
4. Can the same effect be achieved by sampling self. model. num_parallel_samples times with the distribution parameters which is the output of model by single input instead of repeating the input?

[ashok-arjun]

Thanks for elaborating.

1. Yes
2. No, we sample from the distribution, which is a non-deterministic operation even given the same inputs. As we sample, say, 100 samples from the distribution, with the same input, we cover many modes of the distribution, which is what we want.
3. No, please see the answer above
4. It is done to be compatible with the Student's T class that the repetitions are done. You may very well write a Distribution class where the repetitions are done within the class during sampling, but we resorted to the library function for which we had to repeat the inputs manually.

Feel free to follow up! :)

[SpeeeedLee]

@ashok-arjun
May I ask is there a quick way to extract the mean and std of the t-distribution that model predicts?