Feature: Fine-tune TinyLlama and Qwen2.5-coder models for Magistrala and Prism codebase

[drasko]

Is your feature request related to a problem? Please describe.

No

Describe the feature you are requesting, as well as the possible use case(s) for it.

As LLM can fine-tuned on custom data sets, so can SLMs.

We want to fine-tune:

• TinyLlama
• Phi-3

And we want to fine-tune them on our custom Magistrala, Prism and Cocos repositories, so that we can enhance their intelligence for code generation for our purposes.

We want to compare:

• Which is better to fine-tune - more documented, easier, faster, etc ...
• Which shows better result after fine-tuning

Some references:

• https://dongreanay.medium.com/fine-tuning-the-tiny-llama-model-on-a-custom-dataset-9d1ce4eb663d
• https://www.kaggle.com/code/tommyadams/fine-tuning-tinyllama
• https://www.reddit.com/r/LocalLLaMA/comments/16t7la1/fine_tune_llama_for_code_generation_seek/
• https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/finetune-small-language-model-slm-phi-3-using-azure-machine/ba-p/4130399
• https://www.datacamp.com/tutorial/phi-3-tutorial
• https://www.kaggle.com/code/lucamassaron/fine-tune-phi-3-for-sentiment-analysis

Analysis should be done if we should use fine-tuning or RAG for this purpose: https://medium.com/@bijit211987/when-to-apply-rag-vs-fine-tuning-90a34e7d6d25

Indicate the importance of this feature to you.

Must-have

Anything else?

No response

[drasko]

Actually, https://github.com/QwenLM/Qwen2.5-Coder seems more promising - especially that https://ollama.com/library/qwen2.5-coder:1.5b works fast.

We should examine how this one can be fine-tuned as well.

[drasko]

https://qwenlm.github.io/blog/qwen2.5-coder/

• https://debuggercafe.com/fine-tuning-qwen-1-5-for-coding/
• https://medium.com/@danushidk507/qwen2-finetuning-qwen2-f89c5c9d15da
• https://github.com/QwenLM/Qwen/blob/main/finetune.py
• https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat/discussions/6
• https://www.kaggle.com/code/william2020/fine-tune-qwen2-7b-on-custom-data

[drasko]

This probably should be done wia SWIFT, as explained here.

[drasko]

Definitely qwen2.5-coder:1.5b should be set as our default model for now, tested and confirmed by @dborovcanin as well.

@JeffMboya and @rodneyosodo test performance, and if OK, send a PR to replace TinyLlama.