Solve for the reproducibility problem

[Praful932]

Would enable running llmsearch for lightning fast frameworks like exllamav2 , vllm

• The problem with finding good generation parameters for these frameworks is you don’t get the same result twice, due to how the frameworks are implemented. A function that decides in llmsearch which set of generation parameters should be built with determinism in mind. Some ways of solving this would be - letting the framework run a dataset multiple times through it and use mean & standard deviation to judge which set of generation parameters are better. One other aspect that can be thought of is choosing a different set of generation parameters over the current on only when there is significant delta between the two scores.
• There could be other ways that would need to be researched about but the main outcome that solving this issue would drive is, having a better way of performing llmsearch on these fast frameworks.