Merge set_llm() and set_prompt() into one

[maciekbanas]

There is quite a lot of functions now. From user's perspective would it not be simpler, if we passed prompt in set_llm() function? Or process_repos()? @kalimu

[kalimu]

There was a rationale behind the current setup. I've got a few arguments to support it:

1. The set_prompt() currently sets the system prompt only. But I wanted to have it open for different extensions in the future: like setting user prompt (or a part of it), or a prompts that merges partial outputs from different chunks.
2. This is the most important step in the workflow so I would rather keep it separate from the rest workflow steps.
3. We might want to improve prompts in many iterations while the LLM infra backed won't be changing.
4. There is also a possibility that one might want to keep the prompt constant (with some prompt templates for example that we might want to support more explicitly in the future) but changing the LLM infra backend with set_llm() (to adapt to one's company) - so another reason to keep those separate.

[maciekbanas]

Great, thanks for your clarification. Basically, I think it would be a good idea to put in future the user_prompt in set_prompt() and the system prompt (as a framework for user prompts) in set_llm() as it is done similary in elmer where system prompt is called together with chat instance.

And what do you think of putting set_llm() as parameter in initialize_project() - or do we plan to set different llms for the same project?

[kalimu]

Great, thanks for your clarification. Basically, I think it would be a good idea to put in future the user_prompt in set_prompt() and the system prompt (as a framework for user prompts) in set_llm() as it is done similary in elmer where system prompt is called together with chat instance.

Still, my goal is to decouple prompt engineering from setting up IT backend infrastructure.

And what do you think of putting set_llm() as a parameter in initialize_project() - or do we plan to set different llms for the same project?

The same rationale here. I don't want to pack multiple parameters into a single function call that should be doing one thing well and one thing only. Setting up the project metadata is logically different and distinct from setting up LLM backend. We can have a single project when we varying / testing multiple backends. And more infra-related functions could be added like set_vector_database() or set_embeddings_model().

We don't have that many functions in the workflow right now I believe, so I wouldn't be too concerned about explicitly dealing with different workflow steps with the help of different functions. Using more specialized functions calls is usually clearer than using a single function with many parameters giving multiple side effects.