Enable AQUA SDK & CLI to Deploy Fine-Tuned LLMs in Multi-Model Deployment #1175
Conversation
elizjo
requested review from
darenr,
mayoor,
mrDzurb,
VipulMascarenhas,
qiuosier and
ahosler
as code owners
oracle-contributor-agreement
bot
added
the
OCA Verified
|
📌 Cov diff with main: 📌 Overall coverage: |
|
📌 Cov diff with main: 📌 Overall coverage: |
| @@ -311,6 +312,12 @@ def create_multi( | |||
| # "Currently only service models are supported for multi model deployment." | |||
| # ) | |||
|
|
|||
| is_fine_tuned_model = Tags.AQUA_FINE_TUNED_MODEL_TAG in source_model.freeform_tags | |||
There was a problem hiding this comment.
Could you add some comment what we are doing here?
ads/aqua/finetuning/entities.py
Outdated
| @@ -179,3 +185,42 @@ class CreateFineTuningDetails(Serializable): | |||
|
|
|||
| class Config: | |||
| extra = "ignore" | |||
|
|
|||
|
|
|||
| def extract_base_model_ocid(aqua_model: DataScienceModel) -> Tuple[str, str]: | |||
There was a problem hiding this comment.
The name of the function is a bit confusing, it says that it extracts the base model ocid, but returns name and ocid.
Also I think it would be better to create model/utils.py module and move this logic there. Same for the set_finetune_env_var.
ads/aqua/finetuning/entities.py
Outdated
|
|
||
| env_var.update({"FT_MODEL": f"{fine_tune_output_path}"}) | ||
|
|
||
| return env_var |
There was a problem hiding this comment.
looks like we are already modifying the original env_var, probably we don't need to return anything form this function?
|
Could you add more description in the PR? It would be also helpful if you put in the details how the new config will look like. |
|
📌 Cov diff with main: 📌 Overall coverage: |
|
📌 Cov diff with main: 📌 Overall coverage: |
|
📌 Cov diff with main: 📌 Overall coverage: |
This PR aims to add support for using fine-tuned models in a Multi-Model deployment.
When deploying a model group w/ a fine tuned model