update docs for v0.10.0 #205

alexagriffith · 2022-12-01T01:22:14Z

This PR addresses some of the goals in #204 to update and improve our documentation.
These changes are the MVP needed to get version v0.10.0 out.

This PR addresses the following

updating v1/v2 dataplane docs and structure (still need to add examples)
update custom model example with headers input
basic metrics documentation for prometheus & qpext changes

kserve#180 Signed-off-by: agriffith50 <[email protected]>

netlify · 2022-12-01T01:22:23Z

✅ Deploy Preview for elastic-nobel-0aef7a ready!

Name	Link
🔨 Latest commit	`9346498`
🔍 Latest deploy log	https://app.netlify.com/sites/elastic-nobel-0aef7a/deploys/63c812c673ce0c0009ad400f
😎 Deploy Preview	https://deploy-preview-205--elastic-nobel-0aef7a.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

alexagriffith · 2022-12-21T21:50:49Z

docs/modelserving/v1beta1/transformer/torchserve_image_transformer/README.md

@@ -42,24 +42,24 @@ def image_transform(instance):

 # for REST predictor the preprocess handler converts to input dict to the v1 REST protocol dict
 class ImageTransformer(kserve.Model):
-    def __init__(self, name: str, predictor_host: str):
+    def __init__(self, name: str, predictor_host: str, headers: Dict[str, str] = None):


does init also need self.ready? @yuzisun
looking at this for reference - it isn't exactly the same code though... we should probably test this still works soon https://github.com/kserve/kserve/blob/master/python/custom_transformer/model.py

ye we need self.ready, the code snippet is just for demonstrating purpose to show how to override preprocess/postprocess in transformer for REST/gRPC case, the reference points to the actual working example.

ok makes sense, I can add it to avoid confusion?

docs/modelserving/data_plane/data_plane.md

yuzisun · 2022-12-21T23:46:21Z

docs/modelserving/data_plane/data_plane.md

+* V2 added Server Readiness/Liveness/Metadata endpoints
+* V2 endpoint paths contain `/` instead of `:`
+* V2 renamed `:predict` endpoint to `/infer`
+* V2 allows for model versions in the path 


model version is optional, it is up to the model server if it wants to implement it and you can achieve model versioning in different ways e.g external to the model server.

yeah I guess the point I was adding here was that it allows you to add model version in path, which v1 did not

is there something I should change or add here?

maybe path is ambiguous. I meant request path

docs/modelserving/data_plane/v1_protocol.md

docs/modelserving/observability/prometheus_metrics.md

alexagriffith · 2022-12-22T00:33:35Z

docs/admin/serverless/kourier_networking/README.md

@@ -1,5 +1,5 @@
 # Deploy InferenceService with Alternative Networking Layer
-KServe v0.9 and prior versions create the top level `Istio Virtual Service` for routing to `InferenceService` components based on the virtual host or path based routing.
+KServe creates the top level `Istio Virtual Service` for routing to `InferenceService` components based on the virtual host or path based routing.


it seems like saying 0.9 and prior is kind of not useful right? means every version. and it doesn't change with 0.10. so that we dont have to update this with each version update, we can just say kserve creates until that changes.

unless im reading this wrong. the next sentence is that "Now kserve provides an option..." but kserve is at v0.9 so im confused a little by what this means.

should it maybe say "Starting with v0.9, Kserve provides an option...."

I agree with above change. And replacing "Now" with "Starting with v0.9" makes it clearer too when the option to disable was provided.

im not 100% sure this started with v0.9, @yuzisun can you confirm?

docs/modelserving/v1beta1/serving_runtime.md

docs/modelserving/data_plane/data_plane.md

docs/modelserving/data_plane/v1_protocol.md

yuzisun · 2023-01-12T14:42:18Z

docs/modelserving/observability/prometheus_metrics.md

+
+If needed, this value can be overridden in the InferenceService YAML. 
+
+To enable prometheus metrics, add the annotation `serving.kserve.io/enable-prometheus-scraping` to the InferenceService YAML. 


There is also a global config to turn on the scraping without adding this to each inference service yaml right ?

the info for that is in the docs linked below I can add another link here

we expect user to patch the config map themselves right?

docs/modelserving/v1beta1/serving_runtime.md

tessapham

This looks great! The doc is clear enough to me who don't know much about KServe. Only have a few comments on the V2 API.

tessapham · 2023-01-17T19:20:13Z

docs/admin/serverless/kourier_networking/README.md

@@ -1,5 +1,5 @@
 # Deploy InferenceService with Alternative Networking Layer
-KServe v0.9 and prior versions create the top level `Istio Virtual Service` for routing to `InferenceService` components based on the virtual host or path based routing.
+KServe creates the top level `Istio Virtual Service` for routing to `InferenceService` components based on the virtual host or path based routing.


I agree with above change. And replacing "Now" with "Starting with v0.9" makes it clearer too when the option to disable was provided.

docs/modelserving/data_plane/v1_protocol.md

tessapham · 2023-01-17T21:18:58Z

docs/modelserving/data_plane/v2_protocol.md


- GET v2
+<!-- // TODO: add example with -d inputs. -->


Do you still need an example here?

we can, im unsure right now of the best way to add these..

docs/modelserving/data_plane/v2_protocol.md

Signed-off-by: agriffith50 <[email protected]> Signed-off-by: agriffith50 <[email protected]>

Signed-off-by: agriffith50 <[email protected]>

yuzisun · 2023-01-21T00:01:40Z

Awesome work! @alexagriffith

/lgtm
/approve

kserve-oss-bot · 2023-01-21T00:01:47Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alexagriffith, yuzisun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [yuzisun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* clarify runtimes.txt usage in custom model kserve#180 Signed-off-by: agriffith50 <[email protected]> * updating docs to reflect v0.10 Signed-off-by: agriffith50 <[email protected]> Signed-off-by: agriffith50 <[email protected]> * update wording ab model version Signed-off-by: agriffith50 <[email protected]> * add model ready Signed-off-by: agriffith50 <[email protected]> * update serving runtime table Signed-off-by: agriffith50 <[email protected]> * add mlfow to runtime table Signed-off-by: agriffith50 <[email protected]> * update dataplane md Signed-off-by: agriffith50 <[email protected]> * update v1 Signed-off-by: agriffith50 <[email protected]> * fix links Signed-off-by: agriffith50 <[email protected]> * resolving pr comments Signed-off-by: agriffith50 <[email protected]> Signed-off-by: agriffith50 <[email protected]>

clarify runtimes.txt usage in custom model

4646125

kserve#180 Signed-off-by: agriffith50 <[email protected]>

kserve-oss-bot added the do-not-merge/work-in-progress label Dec 1, 2022

kserve-oss-bot requested review from theofpa and xfu83 December 1, 2022 01:22

alexagriffith commented Dec 21, 2022

View reviewed changes

alexagriffith force-pushed the alexagriffith/update_dataplane_docs branch from 63ea55c to d0966b9 Compare December 21, 2022 22:32

alexagriffith marked this pull request as ready for review December 21, 2022 22:35

kserve-oss-bot removed the do-not-merge/work-in-progress label Dec 21, 2022

alexagriffith force-pushed the alexagriffith/update_dataplane_docs branch 5 times, most recently from ae2c6db to cb5484d Compare December 21, 2022 23:35

yuzisun reviewed Dec 21, 2022

View reviewed changes

alexagriffith force-pushed the alexagriffith/update_dataplane_docs branch 2 times, most recently from 3298774 to 78dbe24 Compare December 22, 2022 00:25

alexagriffith commented Dec 22, 2022

View reviewed changes

alexagriffith changed the title ~~Alexagriffith/update dataplane docs~~ update docs for v0.10.0 Dec 22, 2022

alexagriffith mentioned this pull request Dec 22, 2022

add model_ready v2 endpoint kserve/kserve#2617

Merged

5 tasks

yuzisun mentioned this pull request Jan 10, 2023

KServe online documentation for 0.9 release shows non-existent routes kserve/kserve#2637

Closed

alexagriffith force-pushed the alexagriffith/update_dataplane_docs branch from 47b7399 to af604e3 Compare January 10, 2023 21:24

alexagriffith requested review from yuzisun and removed request for theofpa and xfu83 January 10, 2023 21:24

alexagriffith commented Jan 10, 2023

View reviewed changes

docs/modelserving/v1beta1/serving_runtime.md Outdated Show resolved Hide resolved

yuzisun reviewed Jan 12, 2023

View reviewed changes

docs/modelserving/data_plane/data_plane.md Outdated Show resolved Hide resolved

yuzisun reviewed Jan 12, 2023

View reviewed changes

docs/modelserving/data_plane/v1_protocol.md Outdated Show resolved Hide resolved

yuzisun reviewed Jan 12, 2023

View reviewed changes

docs/modelserving/v1beta1/serving_runtime.md Outdated Show resolved Hide resolved

ckadner mentioned this pull request Jan 12, 2023

Update docs for ModelMesh v0.10 #214

Merged

alexagriffith force-pushed the alexagriffith/update_dataplane_docs branch 2 times, most recently from 99d2048 to 7993db1 Compare January 17, 2023 18:32

alexagriffith requested a review from yuzisun January 17, 2023 18:35

tessapham reviewed Jan 17, 2023

View reviewed changes

alexagriffith force-pushed the alexagriffith/update_dataplane_docs branch 2 times, most recently from f4dbc55 to 7831ed6 Compare January 18, 2023 15:38

alexagriffith added 9 commits January 18, 2023 10:38

updating docs to reflect v0.10

03c08f1

Signed-off-by: agriffith50 <[email protected]> Signed-off-by: agriffith50 <[email protected]>

update wording ab model version

654a39e

Signed-off-by: agriffith50 <[email protected]>

add model ready

3ee4078

Signed-off-by: agriffith50 <[email protected]>

update serving runtime table

175aaaa

Signed-off-by: agriffith50 <[email protected]>

add mlfow to runtime table

705924d

Signed-off-by: agriffith50 <[email protected]>

update dataplane md

dc48af1

Signed-off-by: agriffith50 <[email protected]>

update v1

53113aa

Signed-off-by: agriffith50 <[email protected]>

fix links

0f00deb

Signed-off-by: agriffith50 <[email protected]>

resolving pr comments

9346498

Signed-off-by: agriffith50 <[email protected]>

alexagriffith force-pushed the alexagriffith/update_dataplane_docs branch from 7831ed6 to 9346498 Compare January 18, 2023 15:39

yuzisun requested a review from adriangonz January 18, 2023 17:34

kserve-oss-bot assigned yuzisun Jan 21, 2023

kserve-oss-bot added the lgtm label Jan 21, 2023

kserve-oss-bot added the approved label Jan 21, 2023

kserve-oss-bot merged commit 34ddb0a into kserve:main Jan 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update docs for v0.10.0 #205

update docs for v0.10.0 #205

alexagriffith commented Dec 1, 2022 •

edited

Loading

netlify bot commented Dec 1, 2022 •

edited

Loading

alexagriffith Dec 21, 2022

yuzisun Dec 21, 2022

alexagriffith Dec 22, 2022

yuzisun Dec 21, 2022

alexagriffith Dec 22, 2022

alexagriffith Dec 22, 2022

alexagriffith Dec 22, 2022

alexagriffith Dec 22, 2022

alexagriffith Dec 22, 2022

alexagriffith Dec 22, 2022

tessapham Jan 17, 2023

alexagriffith Jan 18, 2023

yuzisun Jan 12, 2023

alexagriffith Jan 17, 2023

alexagriffith Jan 17, 2023

tessapham left a comment

tessapham Jan 17, 2023

tessapham Jan 17, 2023

alexagriffith Jan 18, 2023

alexagriffith Jan 18, 2023

yuzisun commented Jan 21, 2023

kserve-oss-bot commented Jan 21, 2023


		If needed, this value can be overridden in the InferenceService YAML.

		To enable prometheus metrics, add the annotation `serving.kserve.io/enable-prometheus-scraping` to the InferenceService YAML.

update docs for v0.10.0 #205

update docs for v0.10.0 #205

Conversation

alexagriffith commented Dec 1, 2022 • edited Loading

netlify bot commented Dec 1, 2022 • edited Loading

✅ Deploy Preview for elastic-nobel-0aef7a ready!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tessapham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yuzisun commented Jan 21, 2023

kserve-oss-bot commented Jan 21, 2023

alexagriffith commented Dec 1, 2022 •

edited

Loading

netlify bot commented Dec 1, 2022 •

edited

Loading