Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRCC][Ready to Merge] AQUA MS Enhancements #1058

Open
wants to merge 99 commits into
base: main
Choose a base branch
from

Conversation

kumar-shivam-ranjan
Copy link
Member

@kumar-shivam-ranjan kumar-shivam-ranjan commented Feb 5, 2025

Description

This PR intends to enhance the AQUA flow with the introduction of MS enhancements regarding defined and custom metadata artifact APIs.

Related PR

#1036

New APIs introduced

  • Create defined metadata artifact for AQUA unverified model
  • Get defined metadata artifact for AQUA model
  • Changes in model registration in AQUA
  • Changes in evaluation flow in AQUA (downloading evaluation report)
  • Changes in get model config
    • Get Readme
    • Get License
    • Get DeploymentConfiguration
    • Get FinetuningConfiguration
  • Introduced new API to fetch model card

E2E Test results

https://confluence.oraclecorp.com/confluence/display/OCIODSC/%5BAQUA%5D+DRCC+changes+tests+for+AQUA+UI+and+ADS

Unit Tests

Screenshot 2025-03-16 at 5 49 23 AM

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Feb 5, 2025
@kumar-shivam-ranjan kumar-shivam-ranjan changed the base branch from main to feature/aqua_ms_changes_2 February 10, 2025 12:27
@kumar-shivam-ranjan kumar-shivam-ranjan changed the base branch from feature/aqua_ms_changes_2 to main February 10, 2025 12:29
@kumar-shivam-ranjan kumar-shivam-ranjan changed the base branch from main to feature/aqua_ms_changes_2 February 10, 2025 12:53
@kumar-shivam-ranjan kumar-shivam-ranjan changed the title [WIP] ADS changes for AQUA MS enhancements ADS AQUA changes for MS enhancements Mar 3, 2025
@kumar-shivam-ranjan kumar-shivam-ranjan marked this pull request as ready for review March 3, 2025 21:52
@kumar-shivam-ranjan kumar-shivam-ranjan added enhancement New feature or request AQUA labels Mar 27, 2025
Copy link

📌 Cov diff with main:

Coverage-60%

📌 Overall coverage:

Coverage-58.42%

Copy link

📌 Cov diff with main:

Coverage-61%

📌 Overall coverage:

Coverage-58.43%

Copy link

github-actions bot commented Apr 1, 2025

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-19.22%

Copy link

github-actions bot commented Apr 1, 2025

📌 Cov diff with main:

Coverage-57%

📌 Overall coverage:

Coverage-58.50%

)
dsc_model.get_custom_metadata_artifact(EVALUATION_REPORT_MD, temp_dir)
dsc_model.get_custom_metadata_artifact(EVALUATION_REPORT_JSON, temp_dir)
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should try to fetch from the artifacts as well at first, and then form OSS

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I thought i misunderstood this comment.
Not sure if i still understand
For Evaluation , we are fetching the reports first from metadata artifact , if not found ,we fetch from OSS.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh , I see you mean from model artifact? Yes we can do that.
Currently , we check in custom metadata. if not found , we check in customer's OSS bucket where evaluation reports are stored.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. @mrDzurb

@@ -1376,11 +1405,10 @@ def delete(self, eval_id):

@staticmethod
@fire_and_forget
def _delete_job_and_model(job, model):
def _delete_job_and_model(job: DataScienceJob, model: DataScienceModel):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Copy link

github-actions bot commented Apr 3, 2025

📌 Cov diff with main:

Coverage-60%

📌 Overall coverage:

Coverage-58.64%

Copy link

github-actions bot commented Apr 3, 2025

📌 Cov diff with main:

Coverage-62%

📌 Overall coverage:

Coverage-58.65%

mrDzurb
mrDzurb previously approved these changes Apr 3, 2025
@@ -64,6 +64,7 @@
MAX_DISPLAY_VALUES = 10

UNKNOWN = ""
UNKNOWN_LIST = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: I think we don't really need this. We can use the [] directly wherever we need this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. Thanks!

@mrDzurb mrDzurb requested a review from mayoor April 3, 2025 22:25
Copy link

github-actions bot commented Apr 4, 2025

📌 Cov diff with main:

Coverage-60%

📌 Overall coverage:

Coverage-58.63%

@kumar-shivam-ranjan kumar-shivam-ranjan changed the title [DRCC] AQUA MS enhancements [DRCC][READY TO MERGE] AQUA MS Enhancements Apr 4, 2025
@kumar-shivam-ranjan kumar-shivam-ranjan changed the title [DRCC][READY TO MERGE] AQUA MS Enhancements [DRCC][Ready to Merge] AQUA MS Enhancements Apr 4, 2025
Copy link

github-actions bot commented Apr 4, 2025

📌 Cov diff with main:

Coverage-60%

📌 Overall coverage:

Coverage-58.63%

Copy link
Member

@VipulMascarenhas VipulMascarenhas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AQUA enhancement New feature or request OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants